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We consider a class of stochastic processing networks. Assume 
that the networks satisfy a complete resource pooling condition. We 
prove that each maximum pressure policy asymptotically minimizes 
the workload process in a stochastic processing network in heavy traf- 
' fic. We also show that, under each quadratic holding cost structure, 

, there is a maximum pressure policy that asymptotically minimizes 

the holding cost. A key to the optimality proofs is to prove a state 
space collapse result and a heavy traffic limit theorem for the network 
■ processes under a maximum pressure policy. We extend a framework 

of Bramson [Queueing Systems Theory Appl. 30 (1998) 89-148] and 
Williams [Queueing Systems Theory Appl. 30 (1998b) 5-25] from the 
multiclass queueing network setting to the stochastic processing net- 
work setting to prove the state space collapse result and the heavy 
traffic limit theorem. The extension can be adapted to other studies 
of stochastic processing networks. 

(N 



1. Introduction. This paper is a continuation of Dai and Lin (2005), 
in whicii maximum pressure policies are shown to be throughput optimal 
I for a class of stochastic processing networks. Throughput optimality is an 

■ important, first-order objective for many networks, but it ignores some key 

secondary performance measures like queueing delays experienced by jobs 
k>( \ in these networks. In this paper we show that maximum pressure policies 

^ ■ enjoy additional optimality properties; they are asymptotically optimal in 
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minimizing a certain workload or holding cost of a stochastic processing 
network. 

Stochastic processing networks have been introduced in a series of three 
papers by Harrison (2000, 2002, 2003). In Dai and Lin (2005) and this pa- 
per we consider a special class of Harrison's model. This class of stochastic 
processing networks is much more general than multiclass queueing net- 
works that have been a subject of intensive study in the last 20 years; see, 
for example, Harrison (1988), Williams (1996) and Chen and Yao (2001). 
The added features in a stochastic processing network allow one to model 
skills-based routing in call centers [Cans, Koole and Mandelbaum (2003)], 
operator-machine interactions in semiconductor wafer fabrication facilities 
[Kumar (1993)], and combined input- and output-queued data switches 
[Chuang et al. (1999)] in data networks. 

For this general class of stochastic processing networks, Dai and Lin (2005) 
propose a family of operational policies called maximum pressure policies 
and prove that they are throughput optimal. For a given vector a > 0, the 
maximum pressure policy associated with the parameter a is specified in 
Section 3 of Dai and Lin (2005) and will be specified again in Definition 1 of 
this paper. In this paper, for a stochastic processing network that satisfies a 
complete resource pooling condition and a heavy traffic condition, we first 
show in Theorem 2 that a certain workload process is asymptotically mini- 
mized under any maximum pressure policy. When the holding cost rate is a 
quadratic function of the buffer contents, we show then in Theorem 3 that 
there is a maximum pressure policy that asymptotically minimizes the hold- 
ing cost. Our maximum pressure policies do not solve the linear holding cost 
optimization problem for stochastic processing networks. However, following 
an approach in Stolyar (2004), one can find a maximum pressure policy that 
is asymptotically e-optimal under the linear holding cost structure. Section 9 
elaborates the e-optimality of maximum pressure policies. 

In Theorem 2, except for some nonnegativity requirements, the parameter 
a that is used to define a maximum pressure policy can be chosen arbitrarily. 
In Theorem 3, to minimize a given quadratic holding cost rate, one has 
to choose the parameter a to be the vector of coefficients that define the 
quadratic holding cost rate function. In both cases, the parameter can be 
chosen to be independent of network data like arrival or processing rates. 
Thus, these maximum pressure policies do not depend on the arrival rates. 
This feature is attractive in some applications when network data like arrival 
rates are sometimes difficult or impossible to be estimated accurately. The 
maximum pressure policies in Section 9 do depend on the arrival rates of 
the network. 

Our asymptotic region is when the stochastic processing network is in 
heavy traffic; at least one server has to be 100% busy in order to handle all 
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the input. A key assumption on our network is that a complete resource pool- 
ing condition is satisfied. Roughly speaking, the complete resource pooling 
condition requires enough overlap in the processing capabilities of bottleneck 
servers that these servers form a single, pooled resource or "super server." As 
will be discussed fully in Section 3, the complete resource pooling condition 
is articulated by the dual problem of a linear program (LP) called the static 
planning problem. For a network satisfying the complete resource pooling 
condition, the corresponding dual LP has a unique optimal solution and the 
workload process is defined by this unique optimal solution. Ata and Kumar 
(2005) develop a discrete-review policy and prove its asymptotic optimal- 
ity in minimizing the linear holding cost for a class of unitary stochastic 
processing networks that satisfy the complete resource pooling condition 
and the balanced heavy traffic condition. The latter condition requires every 
server in the network to be heavily loaded. This balanced load requirement, 
combined with the complete resource pooling assumption, rules out some 
well-known networks such as multiclass queueing networks, which have been 
used to model semiconductor fabrication lines [Kumar (1993)]. Our defini- 
tion of heavy traffic condition is less restrictive than those in Ata and Kumar 
(2005); our stochastic processing networks include those multiclass queue- 
ing networks that have a unique bottleneck server. Note that the nonbot- 
tleneck stations may not disappear in a heavy traffic diffusion limit in a 
multiclass queueing network operating under a nonidling service policy such 
as first-come-first-serve; see, for example, Bramson (1994). Even under an 
asymptotically optimal maximum pressure policy studied in this paper, the 
non-bottleneck stations may not disappear; see the example below Theo- 
rem 4 in Section 5. 

The major part of our optimality proof of the maximum pressure policies 
is a heavy traffic limit theorem. The theorem asserts that when the network is 
operated under a maximum pressure policy, (a) the one-dimensional work- 
load process converges to a reflecting Brownian motion in diffusion limit, 
and (b) the multidimensional buffer content process is a constant multiple 
of the workload process in diffusion limit. The latter result is a form of 
state space collapse for network processes, and its proof is the key to the 
proof of the limit theorem. We choose to extend a framework of Bramson 
(1998) and Williams (1998b), from the multiclass queueing network setting 
to the stochastic processing network setting, to prove the heavy traffic limit 
theorem. We first show that all solutions to a critically loaded fluid model op- 
erating under a maximum pressure policy exhibit some form of state space 
collapse. Then we translate the state space collapse to the diffusion scal- 
ing following Bramson (1998), proving the state space collapse in diffusion 
limit. Once we have the state space collapse result, we invoke a theorem of 
Williams (1998b) for perturbed Skorohod problems to establish the heavy 
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traffic limit theorem for our stochastic processing network operating under 
the maximum pressure pohcy. 

Stolyar (2004) proves that MaxWeight pohcies asymptotically minimize 
the workload processes in heavy traffic for a generalized switch model that 
belongs to one-pass systems in which each job leaves the system after be- 
ing processed at one processing step. Our Theorem 2 greatly generalizes 
Stolyar (2004) from one-pass systems to stochastic processing networks. Ex- 
cept for Ata and Kumar (2005) that was discussed earlier in this introduc- 
tion, most other works that are closely related to our work have focused on 
parallel server systems. These systems belong to a special class of one-pass 
systems. All these works assume heavy traffic and complete resource pooling 
conditions. For a parallel server system that has 2 buffers, 2 processors and 3 
activities, Harrison (1998) develops a "discrete-review" policy via the BIG- 
STEP procedure that was first described in Harrison (1996) for multiclass 
queueing networks. Furthermore, he proves that the discrete-review policy 
asymptotically minimizes the expected discounted, cumulative linear hold- 
ing cost under the restrictive assumption of Poisson arrival processes and 
deterministic service times. Harrison and Lopez (1999) then use the BIG- 
STEP procedure to produce a family of policies for general parallel systems, 
but they have not proved the optimality of these policies. For the same 2 
buffer, 3 processor and 3 activity parallel server system, but with general 
arrival processes and service time distributions. Bell and Williams (2001) 
develop simple form, buffer priority policies with thresholds and prove their 
asymptotic optimality under linear holding cost. Since the threshold values 
are constantly monitored, these policies are termed as "continuous-review" 
policies. They further generalize their policies to general parallel server sys- 
tems in Bell and Williams (2005) and prove that they are asymptotically 
optimal. While all these works deal with the holding cost objective, the pro- 
posed asymptotically optimal policies in the literature exploit the special 
network structures and critically depend on the network data, particularly 
the arrival rates. Mandelbaum and Stolyar (2004) propose a generalized c// 
policy for parallel server systems. The policy does not use any arrival rate 
information. They prove that it is asymptotically optimal in minimizing a 
strictly convex holding cost. 

When a network has multiple bottlenecks so that the complete resource 
pooling condition is not satisfied, finding an asymptotically optimal policy 
remains a difficult, open problem in general. Shah and Wischik (2006) study 
the fluid and diffusion limits under MaxWeight policies for input-queued 
switches that do not satisfy the complete resource pooling condition. They 
propose a policy that is believed to be asymptotically optimal. However, they 
do not provide a proof for the optimality. Harrison and Wein (1989) study a 
two-station multiclass queueing network known as crisscross network. They 
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propose a threshold type pohcy and demonstrate its near-optimal perfor- 
mance through simulations. For the same crisscross network, but with expo- 
nentially distributed interarrival and service times, Martins, Shreve and Soner 
(1996) and Budhiraja and Ghosh (2005) prove the asymptotic optimality of 
certain nested threshold policies when the network data is in various heavy 
traffic regimes. 

All asymptotic optimality proofs in the literature involve proving a heavy 
traffic limit theorem and some form of state space collapse, either explicitly 
or implicitly. Ata and Kumar (2005) and Bell and Williams (2001, 2005) 
prove the state space collapse results directly without going through fluid 
models. Stolyar (2004) and Mandelbaum and Stolyar (2004) mimic the gen- 
eral framework of Bramson (1998) and Williams (1998b). They start with 
showing a state space collapse result for fluid models, and then prove the op- 
timality directly without proving the state space collapse in diffusion limit as 
an intermediate step. By choosing to extend Bramson and Williams' frame- 
work in this paper, we are able to provide an optimality proof that is clean 
and hopefully easy to follow. We expect our extension can be adapted to 
other studies of stochastic processing networks. Our proof of asymptotic 
optimality requires that the service times have finite 2 + e moments, as in 
Ata and Kumar (2005). This moment assumption is weaker than the expo- 
nential moment assumption that is usually assumed in the literature; see, 
for example, Harrison (1998) and Bell and Williams (2001, 2005). 

Harrison pioneered Brownian control models as a framework to find asymp- 
totically optimal service policies for networks in heavy traffic. The framework 
was first proposed for multiclass queueing networks in Harrison (1988), and 
later was extended for stochastic processing networks in Harrison (2000). In 
his framework, a corresponding Brownian control problem of a stochastic 
processing network is first solved, and then the solution to the Brownian 
problem is used to construct service policies for the original stochastic pro- 
cessing network. Finally, these policies are shown to be asymptotically op- 
timal for the stochastic processing network under a heavy traffic condition. 
A key step to solving the Brownian control problem is to have an equiv- 
alent workload formulation of the Brownian control problem as explained 
in Harrison and Van Mieghem (1997). The "workload process" of the Brow- 
nian control model corresponding to the stochastic processing network in 
this paper, as well as in Ata and Kumar (2005), Stolyar (2004), and Bell 
and Williams (2001, 2005), is one-dimensional. Thus, the Brownian control 
problem has a simple solution. Our maximum pressure policies, at least un- 
der a special linear holding cost structure, can be considered as another 
"interpretation" of the solution to the Brownian control problem, although 
this interpretation is not as direct as those in Bell and Williams (2005) and 
Ata and Kumar (2005). Our paper, together with these papers in the liter- 
ature, demonstrates that the interpretation of the Brownian solution is not 
unique, proving the optimality of the interpreted policies can be difficult. 
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Maximum pressure type of policies were pioneered by Tassiluas and his co- 
authors under various names including back-pressure policies; see, for exam- 
ple, Tassiluas and Ephremides (1992, 1993), Tassiulas (1995) and 
Tassiulas and Bhattacharya (2000). The work of Tassiulas and Bhattacharya 
(2000) represents a significant advance in finding efficient operational poli- 
cies for a wide class of networks, and is closely related to Dai and Lin (2005). 
Readers are referred to Dai and Lin (2005) for an explanation of the major 
differences of these two works. We note that, contrary to the description in 
Dai and Lin (2005), Tassiluas and Ephremides (1992, 1993) and Tassiulas 
(1995) do cover network models, not just one-pass systems. For a recent sur- 
vey of these policies and their applications to wireless networks, see Geor- 
giadis, Neely and Tassiulas (2006). 

The remainder of the paper is organized as follows. In Section 1.1 we col- 
lect some of the notation used in this paper. In Section 2 we describe a class 
of stochastic processing networks, and introduce the maximum pressure ser- 
vice policies. We then define the workload process of a stochastic processing 
network in Section 3, where we also introduce the complete resource pool- 
ing condition. The main results of this paper are stated in Section 4. The 
proofs of the main theorems are outlined in Section 5. A key to the proofs 
of these theorems is a state space collapse result of the diffusion-scaled net- 
work processes under a maximum pressure policy. In Section 6 each fluid 
model solution under a maximum pressure policy is shown to exhibit a state 
space collapse. Section 7 applies Bramson's approach [Bramson (1998)] to 
prove the state space collapse of the diffusion-scaled network processes. The 
state space collapse result is converted into a heavy traffic limit theorem in 
Section 8. The limit theorem is used in Section 5 to complete the proofs of 
the main theorems. In Section 9 we discuss the e-optimality of maximum 
pressure policies. A number of technical lemmas as well as Theorem 1 are 
proved in the Appendix A. 

1.1. Notation. We use M'^ to denote the d-dimensional Euclidean space. 
Vectors in are envisioned as column vectors unless indicated otherwise. 
The transpose of a vector v will be denoted as v'. For v,w W^, v ■ w denotes 
the dot product, and v x w denotes the vector {viwi, . . . ,VdWd)' ■ The max 
norm in R"^ is denoted as |-|, and for a matrix A, we use A to denote the 
maximum absolute value among all components. The Euclidean norm ||-|| in 
is defined by \\v\\ = y/v ■ v. For ri,r2 G M, we use ri V r2 and ri A 7'2 to 
denote the maximum and minimum of ri and r2, respectively. 

We use B'^[0,oo) to denote the set of functions /: [0,oo) ^ that are 
right continuous on [0,oo) having left limits in (0,oo). For / G ©'^[OjOo), we 
let 

\\f\\t = sup |/(.)|. 

0<s<t 
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We endow the function space ©"^[O, oo) with the usual Skorohod Ji -topology 
[Ethier and Kurtz (1986)]. A sequence of functions {/J C B'^[0, oo) is said 
to converge to an / € D'^[0, oo) uniformly on compact (u.o.c.) sets, denoted 
as /r(-) — > /(•), if for each i > 0, limr_^oo \\fr — f\\t = 0. For a sequence of 
stochastic processes {X^} taking values in D'^[0,oo), we use =^ X to 
denote that X"^ converges to X in distribution. 

2. Stochastic processing networks. In this section we describe a general 
stochastic processing network proposed by Dai and Lin (2005). We follow 
the notation of Dai and Lin (2005). The network is assumed to have I + 1 
buffers, J activities and K processors. Buffers, activities and processors are 
indexed by i = 0, . . . , I, j = 1, . . . , J and A; = 1, . . . , K, respectively. For nota- 
tional convenience, we define X = {1, . . . , 1} the set of buffers excluding buffer 
0, J' = {1, . . . ,J} the set of activities and /C = {1, . . . , K} the set of proces- 
sors. Each buffer, with infinite capacity, holds jobs or materials that await 
service. Buffer is a special one that is used to model the outside world, 
where an infinite number of jobs await. Each activity can simultaneously 
process jobs from a set of buffers. It may require simultaneous possession of 
multiple processors to be active. Jobs departing from a buffer will go next to 
other buffers with certain probabilities that depend on the current activity 
taken. 

2.1. Resource consumption. Each activity needs one or more processors 
available to be active. For activity j, A^j = 1, if activity j requires processor 
k, and A^j = otherwise. The K x J matrix A = (Akj) is the resource con- 
sumption matrix. Each activity may be allowed to process jobs in multiple 
buffers simultaneously. For activity j, we use the indicator function Bji to 
record whether buffer i can be processed by activity j. {Bji = 1 if activity 
j processes buffer i jobs.) The set of buffers i with Bji = 1 is said to be the 
constituency of activity j . It is denoted by Bj . The constituency is assumed 
to be nonempty for each activity j G J ^ and may contain more than one 
buffer. When a processing requirement of an activity is met, a job departs 
from each one of the constituent buffers. For each activity j, we use Uj{t}/ jij 
to denote the ^th activity j processing requirement, where Uj = {uj{i),i > 1} 
is an i.i.d. sequence of random variables, defined on some probability space 
(fi, J^,P), and fij is a strictly positive real number. We set (t| = var(nj(l)), 
and assume that aj < oo and uj is unitized, that is, E[nj(l)] = 1, where E is 
the expectation operator associated with the probability measure P. It fol- 
lows that l/nj and aj are the mean and coefficient of variation, respectively, 
for the processing times of activity j. 

An activity j is said to be an input activity if it processes jobs only from 
buffer 0, that is, Bj = {0}. An activity j is said to be a service activity if 
it does not process any job from buffer 0, that is, ^ Bj. We assume that 
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each activity is either an input activity or a service activity. We further 
assume that each processor processes either input activities only or service 
activities only. A processor that only processes input activities is called 
an input processor, and a processor that only processes service activities is 
called a service processor. The input processors process jobs from buffer 
(outside) and generate the arrivals for the network. We denote to be the 
set of input activities, J's the set of service activities, /C/ the set of input 
processors, and ICs the set of service processors. 

2.2. Routing. Buffer i jobs, after being processed by activity j, will go 
next to other buffers or leave the system. Let eo be the I-dimensional vec- 
tor of all O's, and for i £ I, e,, is the I-dimensional vector with the ith 
component 1 and other components 0. For each activity j £ J and each 
constituent buffer i € Bj, we use an I-dimensional binary random vector 
'Pii^) = ("/"i j' W 5 ^' ^ to denote the routing vector of the £th buffer i job 
processed by activity j, where (/>|(^) = ej/ if the ith buffer i job processed 
by activity j goes next to buffer i' , and (pKi) = eo if the job leaves the sys- 
tem. We assume that the sequence (pl = {(pl{^),i > 1} is i.i.d., defined on 
the same probability space {Q,T,¥), for each activity j £ J and i £ Bj. Set 
P-^, = E[(/);^^,(l)]. Then P--, is the probability that a buffer i job processed by 
activity j will go next to buffer i' . 

For each j £ J^i^i Bj, the cumulative routing process is defined by the 
sum 

n=l 

and denotes the number of jobs that will go next to buffer i' among 

the first i buffer i jobs that are processed by activity j. 
The sequences 

{uj,(l)l:ieBj,jeJ) 

are said to be the primitive increments of the network. We assume that they 
are mutually independent and all are independent of the initial state of the 
network. 

2.3. Resource allocations. Because multiple activities may require usage 
of the same processor, not all activities can be simultaneously undertaken 
at a 100% level. Unless stated otherwise, we assume that each processor's 
service capacity is infinitely divisible, and processor-splitting of a processor's 
service capacity is realizable. We use a nonnegative variable aj to denote 
the level at which processing activity j is undertaken. When Oj = 1, activity 
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j is employed at a 100% level. When aj =0, activity j is not employed. 
Suppose that the engagement level of activity j is aj, with < Oj < 1. The 
processing requirement of an activity j job is depleted at rate aj. (The job 
finishes processing when its processing requirement reaches 0.) The activity 
consumes ajAi^j fraction of processor /c's service capacity per unit time. The 
remaining service capacity, 1 — ajA^j, can be used for other activities. 

We use a = (aj) G MlJ, to denote the corresponding J-dimensional alloca- 
tion (column) vector, where IR+ denotes the set of nonnegative real numbers. 
Since each processor k can decrease processing requirements at the rate of 
at most 1 per unit of time, we have 

(2.1) ^ ^kj0.j < 1 for each processor k. 

In vector form, Aa < e, where e is the K-dimensional vector of ones. We 
assume that there is at least one input activity and that the input processors 
never idle. Namely, 

(2.2) ^ ^kjO'j = 1 for each input processor k. 

We use A to denote the set of allocations a G M."^ that satisfy (2.1) and (2.2). 

Each a G A represents an allowable allocation of the processors work- 
ing on various activities. We note that A is bounded and convex. Let £ = 
{a^, . . . , aP} be the set of extreme points of A, where the total number E of 
extreme points is finite. 

Remark. Harrison (2002) does not have the concept of input proces- 
sor and input activity for his stochastic processing networks. The input 
processors and activities in our network model allow us to capture the dy- 
namic routing decisions for external arrivals. (Note that in our model the 
dynamic routing decision of a job is made not at the time of its arrival, but 
at the arrival time of the previous job.) However, if we restrict ourselves 
to the networks without these routing decisions, Harrison's network mod- 
els are broader than ours; Harrison describes his models through first-order 
network data only, leaving the underlying stochastic primitives and system 
dynamics unspecified. In this sense, our network models are a special class 
of Harrison's, which include some stochastic processing networks where the 
service requirements for jobs from different buffers can be different even if 
they are processed simultaneously by a single activity. The latter networks 
are not covered in this paper. 

2.4. Service policies. Each job in a buffer is assumed to be processed by 
one activity in its entire stay at the buffer. A processing of an activity can 
be preempted. In this case, each in-service job is "frozen" by the activity. 
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The next time the activity is made active again, the processing is resumed 
from where it was left off. In addition to the availabihty of processors, a 
(nonpreempted) activity can be made active only when each constituent 
buffer has jobs that are not in service or frozen. We assume that within each 
buffer jobs are queued in the order of their arrivals to the buffer, and head- 
of-line policy is used. When a (nonpreempted) activity becomes active with 
a given engagement level, the leading job in each buffer that is not in service 
or frozen is processed. If multiple activities are actively working on a buffer, 
there are multiple jobs in the buffer that are in service. For an allocation a, if 
there is an activity j with aj > that cannot be made active, the allocation 
is infeasible. At any given time t, we use A{t) to denote the set of allocations 
that are feasible at that time. A service policy specifies which allocation is 
being undertaken at each time t > 0, and we use vr = {vr(t) :t>0} to denote 
such a policy. Under the policy vr, allocation 7r(t) E A{t) will be employed 
at time t. 

Dai and Lin (2005) propose a family of service policies called maximum 
pressure policies that are throughput optimal for a large class of stochastic 
processing networks. To describe these policies for our network, for each 
buffer i = 1, . . . , I and each activity j = 1, . . . , J, we define 



The I X J matrix R= (Rij) is called the input-output matrix in Harrison 
(2002). One interprets Rij as the average amount of buffer i material con- 
sumed per unit of activity j, with a negative value being interpreted to mean 
that activity j is a net producer of material in buffer i. Define £{t) = £riA{t) 
to be the set of feasible extreme allocations at time t. Denote Z = {Z{t),t > 
0} to be the buffer level process with Zi{t) being the buffer level of buffer i 
at time t, including those in service or "frozen" at time t. Now we are ready 
to define maximum pressure service policies for our stochastic processing 
network. Each maximum pressure policy is associated with a vector a G 
with ctj > for each i G T. Recall that for two vectors x,y x xy denotes 
the vector (xiyi, . . .,Xdyd)'- 

Definition 1 (Maximum pressure policies). Given a vector a G with 
> for i G X, a service policy is said to be a maximum pressure policy 
associated with parameter a if at each time t, the network chooses an allo- 
cation a* G argmax„g£;(^) pQ,(a, Z(t)), where Pa(a, Z(t)) is called the network 
pressure with parameter a under allocation a and buffer level Z{t) and is 
defined as 



(2.3) 




(2.4) 



Pa{a,Z{t)) = (q X Z{t))-Ra. 
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When more than one allocation attains the maximum pressure, a tie- 
breaking rule is used. Our results are not affected by how ties are broken. 
However, for concreteness, one can order the extreme allocation set and 
always choose the maximum-pressure allocation with the smallest index. In 
general, a maximum pressure policy can be an idling policy; namely, some 
processors may idle even if there are jobs that they can process. 

In Section 4 we are going to show that every maximum pressure policy 
asymptotically minimizes a certain workload process, and when the stochas- 
tic processing network incurs a certain quadratic holding cost rate, there is 
a maximum pressure policy that is asymptotically optimal. 

Remark. Dai and Lin (2005) associate two parameters, a and (3, with 
each maximum pressure policy and define the network pressure as Pa,/3(a, Z{t)) 
(q X Z{t) — P) ■ Ra. It turns out that the second parameter (3 disappears in 
both fluid and diffusion limits and will have no impact on the asymptotic 
performance. Thus, for notational convenience, we set in this paper /3 = 
and associate each maximum pressure policy with one parameter a. 



3. Workload process and complete resource pooling. We define the work- 
load process through a linear program (LP) called the static planning prob- 
lem and its dual problem. For a stochastic processing network with input- 
output matrix R and capacity consumption matrix A, the static planning 
problem is defined as follows: choose a scalar p and a J-dimensional column 
vector X so as to 



mmimize 



(3.1) 
(3.2) subject to 
(3.3) 

(3.4) 
(3.5) 



P 

Rx = 0, 



1 



for each input processor k, 



AkjXj < p for each service processor k, 
x>0. 



For each optimal solution (p, x) to (3.1)-(3.5), the vector x is said to be 
a processing plan for the stochastic processing network, where component 
Xj is interpreted as the long-run fraction of time that activity j is under- 
taken. Since one of the constraints in (3.4) must be binding for a service 
processor, p is interpreted as the long-run utilization of the busiest service 
processor under the processing plan. With this interpretation, the left-hand 
side of (3.2) is interpreted as the long-run net flow rates from the buffers. 
Equality (3.2) demands that, for each buffer, the long-run input rate to the 
buffer is equal to the long-run output rate from the buffer. Equality (3.3) 
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ensures that input processors never idle, while inequality (3.4) requires that 
each service processor's utilization not exceed that of the busiest service 
processor. The objective is to minimize the utilization of the busiest service 
processor. For future references, the optimal objective value p is said to be 
the traffic intensity of the stochastic processing network. 

The dual problem of the static planning problem is the following: choose 
an I-dimensional vector y and a K-dimensional vector z so as to 



(3.6) 



maximize 



E 

fcG/C/ 



Zk, 



(3.7) subject to 
(3.8) 

(3.9) 
(3.10) 



"^^yiRij < — ^ ZkAkj for each input activity j 
'^^yiRij < ^ Zj^Akj for each service activity j, 



fcG/Cs 

zk>0 



for each service processor k. 



Recall that /C/ is the set of input processors, and ICs is the set of service 
processors. Each pair (y, z) that satisfies (3.7)-(3.10) is said to be a resource 
pool. Component yi is interpreted as the work dedicated to a unit of buffer 
i job by the resource pool, and Zk is interpreted as the relative capacity of 
processor k, measured in fractions of the service capacity of the resource 
pool; for each input processor k, the relative capacity z^ is the amount 
of work generated by input processor k per unit of time. Equality (3.9) 
ensures that the service capacity of the resource pool equals the sum of 
service capacities of all service processors. Constraint (3.8) demands that no 
service activity can accomplish more work than the capacity it consumes. 
Recall that —Rij is the rate at which input activity j generates buffer i 
jobs. For each input activity j, constraint (3.7), which can be written as 
Yl,ie.iVi{~^j) ^ J^keJCi ZkA-kj, ensures that the work dedicated to each unit 
of the activity is no less than that which it generates. The objective is 
to maximize X)fceAC/-^fc' which is the total amount of work generated from 
outside by the input processors per unit of time. A service processor k is 
said to be in the resource pool {y, z) if > 0. Constraint (3.9) is an equality 
instead of an inequality because we do not pose a nonnegativity constraint 
on the variable p in the primal LP; the nonnegativity of p is guaranteed 
from constraint (3.4) and the nonnegativity of A and x. 

A bottleneck pool is defined to be an optimal solution {y*,z*) to the 
dual LP (3.6)~(3.10). Let {p*,x*) be an optimal solution to the primal 
LP, the static planning problem (3.1)-(3.5). From the basic duality theory, 
J2j AkjX* = p* for any service processor k with > 0. It says that all service 
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processors in the bottleneck pool {y*,z*) are the busiest servers under any 
optimal processing plan x* . 

For a bottleneck pool iy*,z*), let W{t) = y* ■ Z{t) for t > 0. Then, W{t) 
represents the average total work of this bottleneck pool embodied in all jobs 
that are present at time t in the stochastic processing network. The process 
W = {W{t),t > 0} is called the workload process of this bottleneck pool. 
Although the workload process of a nonbottleneck resource pool (y, z) can 
also be defined by y ■ Z{t), we will focus on the workload processes of bottle- 
neck pools because bottleneck pools become significantly more important in 
heavy traffic. In general, the bottleneck pool is not unique. However, we as- 
sume all the stochastic processing networks considered in this paper have a 
unique bottleneck pool; namely, they satisfy the following complete resource 
pooling condition. 

Definition 2 (Complete resource pooling condition). A stochastic pro- 
cessing network is said to satisfy the complete resource pooling condition if 
the corresponding dual static planning problem (3.6)-(3.10) has a nonnega- 
tive, unique optimal solution {y*,z*). 

For a processing network that satisfies the complete resource pooling con- 
dition, we define the bottleneck workload process, or simply the workload 
process, of the stochastic processing network to be the workload process of 
its unique bottleneck pool. 

The (bottleneck) workload process defined here is different from the work- 
load process defined in Harrison and Van Mieghem (1997). Their workload 
process is multi-dimensional, with some components corresponding to the 
nonbottleneck pools; it is defined in terms of what they call "reversible dis- 
placements." For the networks where their workload process has dimension 
one, these two definitions of the workload process are consistent. 

Remark. Under certain assumptions including a heavy traffic assump- 
tion that requires all servers in the network be critically loaded, Harrison 
(2000) proposes a "canonical" representation of the workload process for a 
stochastic processing network through a dual LP that is similar to (3.6)- 
(3.10). There, basic optimal solutions to the dual LP were chosen as rows of 
the workload matrix which was used to define the workload process. With- 
out his heavy traffic assumption, his "canonical" choice of workload matrix 
would exclude those nonbottleneck servers. In this case, it is not clear how to 
define a "canonical" representation of the workload process to include those 
nonbottleneck servers. Although for some network examples like multiclass 
queueing networks we can define the workload matrix such that its rows 
are the basic solutions to the dual LP, more analysis is required for general 
stochastic processing networks. 
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4. Asymptotic optimality and the main results. The behavior of the 
buffer level process and the workload process for a stochastic processing 
network under any policy is complex. In particular, deriving closed form ex- 
pressions for performance measures involving these processes is not possible 
in general. Therefore, we perform an asymptotic analysis for stochastic pro- 
cessing networks operating under maximum pressure policies. Our asymp- 
totic region is when the network is in heavy traffic; that is, the offered traffic 
load is approximately equal to the system capacity. Formally, we consider a 
sequence of stochastic processing networks indexed byr=l,2,...;asr— >oo, 
the traffic intensity of the rth network goes to one. We assume that these 
networks all have the same network topology and primitive increments. In 
other words, the matrices A and i?, and the sequences {uj, (pl -.j £ J',i £ Bj) 
do not vary with r. However, we allow the processing rates to change with 
r, and use //^ to denote the processing rate of activity j in the rth network. 
Thus, the traffic intensity p'' for the rth network is the optimal objective 
value of the static planning problem (3.1)-(3.5) with the input-output ma- 
trix = (Rlj) given by R^j = fi'j{Bji — J2i'eBj ^i'i)- assume the following 
heavy traffic assumption throughout this paper. 

Assumption 1 (Heavy traffic assumption). There exists a constant fij > 
for each activity j £ J such that, as r — > oo. 



and, setting R = {Rij) as in (2.3) with /Xj being the limit values in (4.1), 
the static planning problem (3.1)-(3.5) with parameter {R,A) has a unique 
optimal solution {p* ,x*) with p* = \. Furthermore, as r — > oo. 



for some constant 6. 

We define the limit network of the network sequence to be the network 
that has the same network topology and primitive increments as networks 
in the sequence, and has processing rates equal to the limit values pj, given 
in (4.1). Assumption 1 basically means that in the limit network there exists 
a unique processing plan x* that can avoid inventory buildups over time, 
and the busiest service processor is fully utilized under this processing plan. 
Condition (4.2) requires that the networks' traffic intensities approach to 1 
at rate r~^ or faster. 

The heavy traffic assumption is now quite standard in the literature; see, 
for example, Bramson and Dai (2001), Chen and Zhang (2000), Dai and Kurtz 
(1995) and Williams (1996, 1998a) for heavy traffic analysis of queueing 
networks and Ata and Kumar (2005), Harrison (2000), Harrison and Lopez 



(4.1) 



p] Pj 



(4.2) 



r(p" -i)^e 
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(1999), Harrison and Van Mieghem (1997) and Williams (2000) for heavy 
traffic analysis of stochastic processing networks. However, heavy traffic as- 
sumptions in the literature usually assume that, in addition to Assumption 1, 
all service processors are fully utilized. The latter assumption, together with 
a complete resource pooling assumption to be introduced later in this sec- 
tion, rules out some common networks such as multiclass queueing networks. 
In our heavy traffic assumption, only the busiest service processor is required 
to be critically loaded, and some other service processors are allowed to be 
under-utilized. 

The optimal processing plan x* given in Assumption 1 is referred to as 
the nominal processing plan. We use Tj (t) to denote the cumulative amount 
of activity j processing time in [0,t] for the limit network; let T(t) be the 
corresponding J-dimensional vector. Then T{t)/t is the average activity lev- 
els over the time span [0,t]. To avoid a linear buildup of jobs over time in 
the limit network, the long-run average rate (or activity level) that activity 
j is undertaken needs to equal x*, that is, 

(4.3) Ivm. T{t) /t = X* almost surely. 

t— >oo 

There should be no linear buildup of jobs under a reasonably "good" policy. 
A policy is said to be efficient for the limit network if (4.3) holds for the 
network operating under the policy. Since we consider a sequence of net- 
works, we would like to define an analogous notion of a "good" or efficient 
policy for the sequence. One can imagine that under a reasonably "good" 
policy, when r is large, the average activity levels over long time spans must 
be very close to the nominal processing plan x* . To be specific, we define 
the notion of asymptotic efficiency as follows. Let T^{t) be the cumulative 
amount of activity j processing time in [0,t] for the rth network and T^'[t) 
be the corresponding J-dimensional vector. 



Definition 3 (Asymptotic efficiency) . Consider a sequence of stochastic 
processing networks indexed by r = 1,2, . . . , where Assumption 1 holds. A 
policy TT is said to be asymptotically efficient if and only if under policy vr, 
with probability 1, for each i > 0, 

(4.4) T''{rH)/r'^ ^x*t as r ^ oo. 



Equation (4.4) basically says that, under an asymptotically efficient pol- 
icy, the average activity levels over a time span of order are very close 
to the nominal processing plan, so that no linear buildup of jobs will occur 
over the time span of this order. 

Asymptotic efficiency is closely related to the throughput optimality as de- 
fined in Dai and Lin (2005). Fluid models have been used to prove through- 
put optimality of stochastic processing networks operating under a policy. 
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Similarly, the fluid model corresponding to the limit network can be used to 
prove the asymptotic efficiency of a policy for the sequence of networks that 
satisfies Assumption 1. In particular, one can prove that a policy tt is asymp- 
totically efficient if the fluid model of the limit network operating under vr 
is weakly stable. To introduce the first result of this paper, we first define 
the extreme-allocation-available (EAA) condition for a stochastic processing 
network. 

Definition 4 (EAA condition). A stochastic processing network is said 
to satisfy the EAA condition if, for any vector q G M^, there exists an extreme 
allocation a* £ £ such that Ra* ■ q = maxa^s Ra ■ q, and for each buffer i with 
J2j O'jBji > 0, buffer level qi is positive. 

Readers are referred to Section 6 of Dai and Lin (2005) for a detailed 
discussion on the EAA condition; there a class of networks is shown to 
satisfy the EAA condition and a network example is shown not to satisfy 
the EAA condition. 

Theorem 1. Consider a sequence of stochastic processing networks that 
satisfies Assumption 1. If the limit network satisfies the EAA condition, then 
for any a G with a > 0, the maximum pressure policy with parameter a 
is asymptotically efficient. 

Using a fluid model, Dai and Lin (2005) prove in their Theorem 2 that, for 
a stochastic processing network satisfying the EAA condition, a maximum 
pressure policy is throughput optimal. The proof of Theorem 1 uses a fluid 
model that will be introduced in Section 6 and is almost identical to that of 
Theorem 2 in Dai and Lin (2005). It will be outlined in Appendix B. 

Asymptotic efficiency helps to identify reasonably "good" policies, but it 
is not very discriminating. We would like to demonstrate a certain sense of 
optimality for maximum pressure policies in terms of secondary performance 
measures. For this, we will introduce two notions of asymptotic optimality. 
Before giving their definitions, we make the following assumption on the 
sequence of networks. 

Assumption 2 (Complete resource pooling). All networks in the se- 
quence and the limit network satisfy the complete resource pooling condition 
defined in Section 3. Namely, the dual static planning problem (3.6)-(3.10) 
of the rth network has a nonnegative, unique optimal solution (y^,z^), and 
the dual static planning problem of the limit network also has a nonnegative, 
unique optimal solution (y*,z*). 

Lemma 1. Assume Assumptions 1 and 2. Then (j/^, z^) — > (y*, z*) as 
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The proof of Lemma 1 will be provided in Appendix B. 

Remark. In Assumption 2 we assume all networks in the sequence sat- 
isfy the complete resource pooling condition so that the workload processes 
can be uniquely defined as in Section 3 by the first order network data 
{K^jA^). This assumption can be removed if one defines the workload pro- 
cess of a network with multiple bottleneck pools to be the workload process 
of an arbitrarily chosen but prespecified bottleneck pool (with y*" being any 
given optimal solution to the dual problem). On the other hand, the complete 
resource pooling condition for the limit network is crucial for our results to 
hold. 

The first notion of asymptotic optimality is in terms of the workload 
process introduced in Section 3. Under Assumption 2, we can define the 
one-dimensional workload process of the rth network as 

(4.5) W^{t)=y'-Z^\t), 

where = {Z^{t)^t> 0} is the buffer level process of the rth network. 

Define the diffusion-scaled workload and buffer level processes of the rth 
network W"" = {W''{t),t> 0} and Z"" = {Z''{t),t> 0} via 

VF-(t) = W^{rH)/r, Z'\t) = Z\rH)lr. 

Clearly, t?^(t) = f ■ Z'\t) for t > 0. 

Definition 5. Consider a sequence of stochastic processing networks 
indexed by r. An asymptotically efficient policy vr* is said to be asymptot- 
ically optimal for workload if for any t > 0,w > 0, and any asymptotically 
efficient policy vr, 

(4.6) lim supF{W^,{t)>w) < lim inf F{W^{t)>w), 

r — >oo ^ 

where WJ^*{-) and W^(-) are the diffusion-scaled workload processes under 
policies vr* and vr, respectively. 

The second notion of asymptotic optimality is in terms of a quadratic 
holding cost structure for the sequence of stochastic processing networks. 
Let h = {hi'.i £2) be a constant vector with hi > for i Gl. For the rth 
network, the diffusion-scaled holding cost rate at time t is 



(4.7) 



H^{t) = Y,hiZlit)f. 
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Definition 6. Consider a sequence of stochastic processing networks 
indexed by r. An asymptotically efficient policy vr* is said to be asymptoti- 
cally optimal for the quadratic holding cost if for any t > 0,rj > 0, and any 
asymptotically efficient policy vr, 

(4.8) limsupP(i?;*(t) > 7?) < liminfP(^;(t) > ??), 

r — ^oo ^ 

where H^*{t) and H^{t) are the diffusion-scaled total holding cost rates at 
time t under policies vr* and vr, respectively. 

To state our main theorems, we make two more assumptions on the se- 
quence of networks. One is a moment assumption on the unitized service 
times Uj{i) and the other is an assumption on the initial buffer level pro- 
cesses. 

Assumption 3. There exists an £„ > such that, for all j, 

(4.9) E[(uj(l))2+^"] <cx). 

Assumption 4 (Initial condition). As r ^ oo, 

(4.10) Z^(0) = Z'^(0)/r ^0 in probability. 

Assumption 3 requires that the unitized service times have finite 2 -t- 
moments. It was used by Ata and Kumar (2005) and it is stronger than some 
standard regularity assumptions such as in Bramson (1998). Assumption 3 
will be used in Section 7 to prove a state space collapse result for stochastic 
processing networks operating under maximum pressure policies. Assump- 
tion 4 holds if the initial buffer levels of the networks are stochastically 
bounded, namely, 

lim limsupP(|Z''(0)| >r) =0. 

Clearly, Assumption 4 implies that, as r — > co, 

(4.11) ^^'■(0)^0 in probability. 

Theorem 2. Consider a sequence of stochastic processing networks. As- 
sume Assumptions 1-4 o.nd that the limit network satisfies the EAA condi- 
tion. Each maximum pressure policy is asymptotically optimal for workload. 

Theorem 3. Consider a sequence of stochastic processing networks where 
Assumptions 1~4 hold and the limit network satisfies the EAA condition. The 
maximum pressure policy with parameter a = h is asymptotically optimal for 
the quadratic holding cost in (4-7). 
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Theorem 2 says that, at every time t, asymptotically, the diffusion-scaled 
workload under any maximum pressure policy vr* is dominated by that under 
any other asymptotically efficient policy vr in the sense of stochastic ordering. 
Theorem 3 says that, given a quadratic holding cost rate structure with 
coefficient vector h, the quadratic holding cost rate at every time t under the 
maximum pressure policy with parameter a = his asymptotically dominated 
by that under any other efficient policy in the sense of stochastic ordering. 

The proofs of Theorems 2 and 3 will be outlined in Section 5. Throughout 
this paper, we shall assume Assumptions 1-4 and that the limit network 
satisfies the EAA condition. 

When the objective is to minimize a linear holding cost, our maximum 
pressure policies do not achieve the asymptotic optimality. However, some 
maximum pressure policies are asymptotically near optimal. In Section 9 we 
introduce the notion of e-optimality and identify a set of maximum pressure 
policies that are asymptotically e-optimal in terms of minimizing the linear 
holding cost. 

5. An outline of the proofs. This section outlines the proofs of our main 
theorems, Theorems 2 and 3. We first derive an asymptotic lower bound on 
the workload processes under asymptotically efficient policies. Then we state 
a heavy traffic limit result in Theorem 4, which implies that this asymptotic 
lower bound is achieved by the workload process under any maximum pres- 
sure policy. The heavy traffic limit theorem also implies the optimality of 
a certain maximum pressure policy under the quadratic holding cost struc- 
ture in Theorem 3. At the end we outline a proof for the heavy traffic limit 
theorem. The key to the proof is a state space collapse result to be stated 
in Theorem 5. 

We first derive an asymptotic lower bound on the workload processes 
under asymptotically efficient policies. That is, we search for a process W* 
such that, under any asymptotically efficient policy, 

liminf P(T?Vi) >w)> W{W*it) > w) for aU t and w. 

r — >oo 

As before, we assume that the sequence of stochastic processing networks 
satisfies Assumption 2. Hence, the one dimensional workload process is 
well defined by (4.5) for each r. 

We begin the analysis by defining a process = {Y^{t),t > 0} for the 
rth network via 

(5.1) y{t) = {l-p')t-y'.R'T'{t). 

Since is interpreted as the traffic intensity of the bottleneck pool, for each 
t>0, p^t is interpreted as the average total work contributed to the bottle- 
neck pool from the exogenous arrivals in [0,t], and (1 — p'^)t represents the 
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average total work that could have been depleted by time t if the bottleneck 
pool never idles. Because of the randomness of the processing times, the bot- 
tleneck pool will almost surely incur idle time over time, particularly when 
the system is not overloaded. Under a service policy and its corresponding 
activity level process , the average total work that has been depleted by 
time t is given by 

j€Js i&I j&Ji «ex 

Note that, as in Section 3, for each service activity j G Js, J2i£iyi^ij is the 
average work accomplished per unit of activity j, and that for each input 
activity j S J/, J2i£i Hi {~ ^ij) is the average work generated per unit of ac- 
tivity j. Therefore, Y^(t) represents the deviation of the workload depletion 
in [0,t] from that under the "best" policy. The following lemma says that 
this deviation does not decrease over time. 

Lemma 2. Consider a sequence of networks satisfying Assumption 2. 
For each r and each sample path, the process defined in (5.1) is a non- 
decreasing function with y (0) = 0. 

We leave the proof to Appendix B. 

Remark. Some special stochastic processing networks, such as multi- 
class queueing networks [Harrison (1988)] and unitary networks [Bramson 
and Williams (2003)], have no control on the input activities. Then, TJ(t) is 
fixed for all j G Ji under different policies, and J2jeJi Tj{t) J2iei Vi i~ -^ij) ~ 
p^t. For these networks, one gets 

Y^{t)=t-Y.Tj{t)Y^ylR:^, 

and Y'^[t) is interpreted as the cumulative idle time of the bottleneck pool 
by time t. 

For each activity j, we define the process = {S^^{t),t > 0} by 

5} (t) = maxj n : J2 % (^) ^ | ■ 

Under a head-of-line service policy, Sj{t) is the number of activity j process- 
ing completions in t units of activity j processing time for the rth network. 
Then we can describe the system dynamics of the rth network by the fol- 
lowing equation: 

z:{t) = zi\o) + Y: E '^usjiTjm 

(5.2) 

- J2 Sj{TJ{t))Bji for each i > and i G J. 
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Since quantity rj(t) is the cumulative amount of activity j processing time 
in [0,t], Sj{TJ{t)) is the number of activity j processings completed by 
time t, and J2j£j 'Sj{Tj{t))Bji is the total number of jobs that depart from 

buffer i gTU {0} in [0,t]. For each activity j, Ei'eBj ^iii^jiTJ {t))) is the 
total number of jobs sent to buffer i by activity j from its constituent buffers 
by time t, so J2j<£jJ2i'<£B is the total number of jobs that go 

to buffer i by time t. Equation (5.2) says that the number of jobs in buffer 
i at time t equals the initial number plus the number of arrivals minus the 
number of departures. 

From (5.2), we can write the workload process = ■ Z'^ as 

W^-{t) = W^{0)+Y^y: ( 4^iS'JiTJm-B,,S^^{TJit))]. 

iei jeJ\i'eBj I 

Let X''(i) = Wit) - Y''{t) for each t > 0. Then 

iex jej Ki'eBj ) 
-{I- p')t + y'- -EJ-T-it). 
We define the following diffusion-scaled processes: 
(5.3) S'j{t) = r-\S'^{rh) - iJi]rh] for each j G J, 

(5.4) 

for each j £ J',i G Bj and i £l, 

(5.5) X'-{t)=r~^X'-{rh), 

(5.6) Y'\t)=r-^Y'{rh). 

In (5.4) [t\ denotes the greatest integer number less than or equal to the 
real number t. ^ 

Then the diffusion-scaled workload process can be written as a sum 
of two processes 

(5.7) W'{t)=X^it) + Y^{t), 
and 

x^(t) = w^[o) + E E f E ^^^lis^iTjm 



(5.8) 

-r{l-pnt, 



\i'PB, I ' 
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where 



f"j{t)=r~^Tj{rH) and S'j (t) = S^^ {rh) . 



The process X^' depends on the pohcy only through the fluid-scaled pro- 
cess T^' . In fact, from Lemma 4.1 of Dai (1995) and (4.4), it follows that, 
under any asymptotically efficient policy, T^— >2;*(-) u.o.c. almost surely, 
where x*{t) = x*t and x* is the optimal solution to the static planning 
problem (3.1)-(3.5) of the limit network. As a consequence, converges 
in distribution to a one-dimensional Brownian motion that is independent 
of policies. 

Lemma 3. Consider a sequence of stochastic processing networks operat- 
ing under an asymptotically efficient policy. Assume Assumptions 1-4. Then 
X^ ^ X* , where X* is a one- dimensional Brownian motion that starts from 
the origin, has drift parameter 9 given in (4-2), and has variance parameter 



Proof. First, Lemma 4.1 of Dai (1995) and (4.4) implies that T (•) 
x*{-) almost surely under any asymptotically efficient policy. Then, the re- 
sult in the lemma follows from (5.8), the functional central limit theorem for 
renewal processes [cf. Iglehart and Whitt (1970)] the random time change 
theorem [cf. Billingsley (1999), (17.9)], and the continuous mapping theo- 
rem [cf. Billingsley (1999), Theorem 5.1]. Deriving the expression for cj^ is 
straightforward but tedious; the derivation is outlined in Harrison (1988) for 
multiclass queueing networks, so we will not repeat it here. □ 

We define the one-dimensional refiection mapping ^/;:D[0,oo) — > D[0, oo) 
such that, for each / G D[0,oo) with /(O) > 0, 



Applying diffusion scaling to Lemma 2, we know that l^^(-) is a nonnega- 
tive, nondecreasing function, so, from (5.7) and the well-known minimality of 
the solution of the one-dimensional Skorohod problem [cf. Bell and Williams 
(2001), Proposition B.l], 




with T^'\j G J,i 



G Bj , defined by 




ifii = i2, 
ifh / *2- 



^(/)(i) = /(i)- inf (/(s)AO). 

0<s<t 



W^{t)>i;{X^-){t) 



for every t and every sample path; 
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namely, ■ip{X'^){t) is a pathwise lower bound on W^. It then follows that 
liminf PfH^^t) > w) > liminf P(V'(X'^)(t) > w) for aU t and w. 

r — ^oo r — ^oo 

Define 

(5.9) W*='ilj{X*). 

Then W* is a one-dimensional reflecting Brownian motion associated with 
X* . Because X^ X* , by the continuous mapping theorem, we have 

ijiX'') ^ W*. 

Because W*{t) has continuous distribution for each t, we have 

lim F(ip(X'')(t) >w)= F(W*(t) > w). 

r — ^oo 

Therefore, 

(5.10) liminf P(W^7*) > "i^) > F(W*{t) > w) for each t and w. 

r — i-oo 

So far, we have shown that W* is an asymptotic lower bound on the work- 
load processes under asymptotically efficient policies. The following heavy 
traffic limit theorem ensures that the workload processes under any max- 
imum pressure policy converge to W*. This completes the proof of Theo- 
rem 2. 



Theorem 4. Consider a sequence of stochastic processing networks op- 
erating under a maximum pressure policy with parameter a > 0. Assume 
Assumptions 1-4 and that the limit network satisfies the EAA condition. 
Then 

(5.11) {W\Z') (VF*,Z*'") asr^oo, 
where W* is given by (5.9) and 

(5.12) Z*''' = CW*, 
with being defined as 



(5.13) C 



Ei'(y*')V"i' 



An outline for proving Theorem 4 will be presented at the end of this sec- 
tion. The full proof of Theorem 4 will be completed in Section 8. Theorem 4 
is known as a heavy traffic limit theorem. It states that the scaled buffer level 
process and the scaled workload process jointly converge in heavy traffic as 
displayed in (5.11), and the limit processes exhibit a form of state space 
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collapse: in diffusion limit, the I-dimensional buffer level process is a con- 
stant vector multiple of the one-dimensional workload process as displayed 
in (5.12). Because buffers at nonbottleneck pools may also contribute to the 
workload process (at the bottleneck pool), the diffusion-scaled buffer level 
processes in these buffers do not go to zero in heavy traffic limit, due to 
the idling nature of our maximum pressure policies. For example, consider 
a network of two stations in series, known as the tandem queue network. 
Suppose that the second station is critically loaded and the first station is 
underloaded. Under any maximum pressure policy, by Theorem 4, the diffu- 
sion limit for the buffer level process at the first station is a one-dimensional 
reflecting Brownian motion. This result is in sharp contrast to the ones in 
Chen and Mandelbaum (1991), in which they prove buffers at nonbottleneck 
stations in a single-class network disappear in heavy traffic diffusion limit 
under the first-come-first-serve nonidling policy. 

With Theorem 4, we now give the proof of Theorem 3. 

Proof of Theorem 3. We first derive an asymptotic lower bound on 
the quadratic holding cost rate process under any asymptotically efficient 
policy. Consider the following quadratic optimization problem: 



The problem (5.14)-(5.15) can be solved easily. The optimal solution is given 



(5.14) 




(5.15) 




by 




and the optimal objective value is given by 




Then under any policy, for all r, we have 




For any asymptotically efficient policy vr 



liminfP(K(t) > 7?) > liminfP (|y;(t))7 E(yr')V/ii' > ^ 





>F{g*iW*{t))>r]), 
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where g*{w) = iv^ / eiiVi')'^ / ^i') ■ The second mequahty follows from (5.10) 
and the fact that ^ y* as r — > cx). 

Now consider the maximum pressure policy vr* having the parameter a = 
h. Prom Theorem 4, we have 

H^t asr— i'oo. 

Because 

K{Zl^\t)f = J2 hiiCl'W^t))' = g*{W*{t)), 

i i 

we have 

\mi¥{Hl,{t) > ri)=¥{g*{W\t)) > r?), 
thus proving inequality (4.8). □ 

The key to the proof of Theorem 4 is the following state space collapse 
result. 

Theorem 5 (State space collapse). Consider a sequence of stochastic 
processing networks operating under the maximum pressure policy with pa- 
rameter a. Assume Assumptions 1-4 and that the limit network satisfies the 
EAA condition. Then, for each T > 0, as r ^ oo, 

(5.16) \\Z''{-)-CW'\-)\\t^^ in probability. 

Recall that is the uniform norm over [0, T]. [The readers should not 
confuse the symbols T and T(-) with one another. We will always include 
"(•)" when dealing with the cumulative activity level process T{-).] Theo- 
rem 5 states a form of state space collapse for the diffusion-scaled network 
process: for large r, the I-dimensional diffusion-scaled buffer level process 
is essentially a constant vector multiple of the one-dimensional workload 
process. 

The proof of Theorem 5 is lengthy. To prove it, we generalize a frame- 
work of Bramson (1998) from the multiclass queueing network setting to the 
stochastic processing network setting. The framework consists of two steps 
that we will follow in the next two sections: First, in Section 6 we will show 
that any fluid model solution for the stochastic processing networks under a 
maximum pressure policy exhibits some type of state space collapse, which 
is stated in Theorem 6 in that section. Then, in Section 7 we will follow 
Bramson's approach to translate the state space collapse of the fluid model 
into a state space collapse result under diffusion scaling, and thus prove 
Theorem 5. 

Once we have Theorem 5, we apply a perturbed Skorohod mapping theo- 
rem from Williams (1998b) to complete the proof of Theorem 4 in Section 8. 
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6. The fluid model. In this section we first introduce the fluid model of 
a sequence of stochastic processing networks operating under a maximum 
pressure pohcy. We then show that any fluid model solution under a maxi- 
mum pressure policy exhibits a form of state space collapse. 

The fluid model of a sequence of stochastic processing networks that sat- 
isfies Assumption 1 is defined by the following equations: for all t > 0, 

(6.1) Z{t) =Z{0)-Rf{t), 

(6.2) Z{t) >0, 

(6.3) Y,Akj{f,{t)-f,{s)) = t-s 

for each < s < t and each input processor fc, 



(6.4) Y.^kj{f,{t)-f,{s)) <t-s 

for each < s < t and each processor A;, 

(6.5) T is nondecreasing and T'(O) = 0. 

Equations (6.1)-(6.5) define the fiuid model under any given service pol- 
icy. Any quantity {Z,T) that satisfies (6.1)-(6.5) is a fluid model solution 
to the fluid model that operates under a general service policy. Following its 
stochastic processing network counterparts, each fluid model solution {Z,T) 
has the following interpretations: Zj{t) the fluid level in buffer i at time t 
and Tj{t) the cumulative amount of activity j processing time in [0, t]. 

Under a specific service policy, there are additional fluid model equations. 
For a given parameter a > 0, we are to specify the fluid model equation as- 
sociated with the maximum pressure policy with parameter a. To motivate 
the fluid model equation, we note that for each fluid model solution {Z,T), it 
follows from equations (6.3)-(6.4) that T, and hence Z, is Lipschitz contin- 
uous. Thus, the solution is absolutely continuous and has derivatives almost 
everywhere with respect to the Lebesgue measure on [0, oo). A time t>0 
is said to be a regular point of the fluid model solution if the solution is 
differentiable at time t. For a function /:M+ — > M*^, where d is some posi- 
tive integer, we use f{t) to denote the derivative of / at time t when the 
derivative exists. From (6.3)-(6.4), one has T{t) G ^ at each regular time t. 
Thus, 

Rf{t) ■ (a X Z{t)) < maxRa ■ (a x Z{t)) = maxRa ■ (a x Z{t)). 

Assume that the limit network satisfies the EAA condition. The fiuid model 
equation associated with the maximum pressure policy with parameter a 
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takes the following form: 

(6.6) RT{t) ■ {a X Z{t)) = m&xRa ■ (a x Z{t)) 

for each regular time t. 

Each fluid model equation will be justified through a fluid limit procedure. 
Two types of fluid limits are considered in this paper. One will be introduced 
in Section 7 and the other in Appendix B. They both satisfy the fluid model 
equations (6.1)-(6.6). Equation (6.6) says that, under the maximum pressure 
policy with parameter a, the instantaneous activity allocation T{t) in the 
fluid model maximizes the network pressure, Ra ■ (a x Z{t)), at each regular 
time t. Any fluid model solution that satisfies fluid model equations (6.1)- 
(6.6) is called a fluid model solution under the maximum pressure policy with 
parameter a. 

Theorem 6. Let {Z,T) be a solution to the fluid model equations (6.1)- 

(6.6) , where a > 0, and {R,A) is the first order network data for the limit 
network that satisfies Assumptions 1 and 2. Suppose \Z[{S)\ < 1. Then there 
exists some finite tq > 0, which depends on just a, I, R and A, such that, 

(6.7) \Z{t) -CW{t)\=0 for all t> To, 

where W = y* ■ Z is the workload process of the fluid model and C,^ is given 
by (5.13). Furthermore, if 

\Z{ti)-CW{ti)\=0 for some Ti>0, 

then 

\Z{t) -CW{t)\=0 forallt>Ti. 

Theorem 6 says that the fluid model under the maximum pressure policy 
exhibits a form of state space collapse: after some finite time tq, the I- 
dimensional buffer level process Z equals a constant vector multiple of the 
one-dimensional workload process W; if this happens at time ri, it happens 
at all times after n. In particular, if Z(0) = C"W^(0), then Z{t) = C"l^(i) 
for all t > 0. 

The rest of this section is devoted to the proof of Theorem 6. We first 
define 

z*it) = cw{t), 

and we shall prove Z{t) — Z*{t) = for t large enough. The following lemma 
will be used in the proof. 
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Lemma 4. For any t>0, 

(6.8) {Z{t)-Z*{t))-y* = 0, 

(6.9) {Z{t)-Z*{t))-{axZ*{t))=0, 
and for each regular time t, 

(6.10) iZ{t)-Z*{t))-{ax2*{t)) = 0. 

Proof. Equality (6.8) follows because 

Z*{t) ■ y* = W{t)C ■ y* = W{t) = Z(t) ■ y*. 
Because Z* (t) = (t) , 

(6.11) a X Z*{t) = W{t)y*/ (j2{y*f/a,^ . 

Then equality (6.9) follows from (6.8). 

Finally, equality (6.10) follows immediately from (6.9). □ 

Define V = {Ra : a S A}. Recall that A is the set of all possible allocations 
and the vector Ra is the average rate at which material is consumed from 
all buffers under allocation a, so y is the set of all possible flow rates out of 
buffers in the limit network. It is obvious that y is a polytope containing the 
origin because Rx* = 0, where, as before, x* is the optimal solution to the 
static planning problem of the limit network. Furthermore, Corollary A.l in 
Appendix A, applied to the limit network, implies that 

(6.12) maxy*-v = 0. 

Since y* ■ Ra is the rate at which the workload is reduced under allocation 
a, therefore, (6.12) says that no feasible allocation can reduce the system 
workload for the limit network, which is not surprising because the limit 
network is in heavy traffic. 

Define a (I — l)-dimensional hyperplane 

(6.13) V° = {veR^:y*-v = 0}. 

The hyperplane V° is a supporting hyperplane of V. Let V* = V DV". Then 
from (6.12) and (6.13), V* = argmax^gy y* • v; namely, V* is the set of all 
possible flow rates that maximize y* ■ v. Since the origin is in both V and 
V°, V* is not empty. Moreover, the following lemma says that V* contains 
a certain (I — 1) -dimensional neighborhood of the origin. 

Lemma 5. There exists some 6 > such that {v £ V° : ||u|| < 5} C y. 
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We provide a proof of Lemma 5 in Appendix B. A key to the proof is 
the complete resource pooUng assumption for the hmit network; namely, 
the dual static planning problem (3.6)-(3.10) has a unique optimal solution. 
The uniqueness of y* ensures that the origin lies in the relative interior of 
one of the facets of V. 

Proof of Theorem 6. Let {Z,T) be a solution to the fluid model 
equations (6.1)-(6.6). We consider the Lyapunov function 

f{t) = {ax{Z{t)-Z*m-{Z{t)-Z*{t)). 

Let v{t) = RT{t); for each buffer i, Vi{t) can be interpreted as the net flow 
rate out of buffer i at time t {total departure rate minus total arrival rate). 
Then it follows from (6.1) that Z{t) = —v{t), and we have 

f{t) = 2{ax{Z{t)-Z*{t))).{Z{t)-Z{t)) 

(6.14) 

= 2{ax{Z{t)-Z*m-{-v{t)). 

The second equality in (6.14) follows from (6.10) in Lemma 4. 

Because f{t) £ A, we have v{t) £ V. Thus, y* ■ v{t) < 0. This, together 
with (6.11), implies that (a x Z*{t)) ■v{t) < 0. Furthermore, the fluid model 
equation (6.6) implies that 

{a X Z{t)) ■ v{t) = max (a x Z{t)) ■ v. 
Therefore, the last term in (6.14) is bounded from above as follows: 



2(a X {Z{t) - Z*{t))) ■ i-v{t)) < -2{a x Z{t)) ■ v{t) 

(6.15) 

= — 2max(Q; X Z{t)) ■ v. 

Since C F, we have 

max(a x Z{t)) ■ v > max(a x Z(t)) ■ v 

v£V v£V* 

(6.16) 

= max(ax {Z (t) - Z* (t))) ■ v . 

The second equality in (6.16) holds because of (6.11) and the fact that 
y* ■v = ior all v G V* . 
If f{t) > 0, let 

, 6{Z{t)-Z*{t)) 

\\zit)-z*{t)W 
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Then \\v*\\ = 6 and y* ■ v* = 0. The latter fact follows from (6.8). Pick 6 as 
in Lemma 5 and it follows from Lemma 5 that v* £V* . Therefore, 

max(a x - Z*{t))) ■ v > {a x {Z{t) - Z*{t))) • v* 

(6.17) 

= Sf{t)/\\Z{t)-Z* 

Combining (6.14)-(6.17), we have 



(6.18) fit) < -26fit)/\\Z{t) - Z*it)\\ < -26^^^,^f{t). 
Therefore, 

f{t) = for t > y/fiO)/ {5^/mmi^J ai). Set tq = maxj£ja./((5Vminjgxai)- 
Then f{t) = for t > tq, because 

/(O) = a X (Z(0) - Z*(0)) • (Z(0) - Z*(0)) = a x (Z(0) - Z*(0)) • Z(0) 
< maxaj||Z(0)||^ < I maxa^. 

Here tq depends only on a, R, A and I, because the set V is completely 
determined by R and A and so is 5. 

Equation (6.18) also implies that for any ri, if /(ri) = 0, then f{t) = for 
alH>Ti. □ 

7. State space collapse. In this section we translate the state space col- 
lapse result of the fluid model into a state space collapse result under the 
diffusion scaling, thus proving Theorem 5. We apply Bramson's approach 
[Bramson (1998)] to prove that, for each T > 0, as r — > oo, 

WZ^'i-) - CW'^{-)\\t ^0 in probability. 

In Bramson's approach, the following fluid scaling plays an important role 
in connecting Theorem 6 with Theorem 5: For each r = 1, 2, . . . , and m = 
0,1,..., 

Sj'"'{t) = -^{Sj{rm + ir,mt) - Sj{rm)) for each j G J, 

for each j £ J',i £ Bj, 
T;'™(i) = -^(T;(rm + ^r,mt) - T]{rm)) for each j G J, 

Zl'"^{t) = - — Zl{rm + ir^mt) for each i G X, 
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where ^r,m = \Z'^{rm)\ V r. Recall that [t\ denotes the integer part of t. 

Here scaling the processes by (^r,m ensures |Z'''™(0)| < 1, which is needed 
for compactness reasons. Using index (r, m) allows the time scale to expand; 
we will examine the processes over [0, L] for m = 0, 1, . . . , \rT~\ — 1, where 
L > and T > are fixed. Thus, the diffusion-scaled time [0, T] is covered 
by \rT~\ fluid scaled time pieces, each with length L. Here [t] denotes the 
smallest integer greater than or equal to t. For future references, we assume 
that L > To V 1, where tq is the time when state space collapse occurs in the 
corresponding fluid model as determined in Theorem 6. 

We outline the proof of Theorem 5 as follows. First, in Proposition 1 be- 
low, we give a probability estimate on the upper bound of the fluctuation of 
the stochastic network processes W'"^{-) = (T'''™(-), Z^' '"(•)). The estimates 
on the service processes 5'J'™'(-) and the routing processes $^''^''"(-) are also 
given. From Proposition 1, a so-called "good" set Q"^ of sample paths can 
be defined, where the processes X'''™ perform nicely for r large enough. On 
this "good" set, for large enough r, the processes X^'"^ can be uniformly ap- 
proximated by so-called Lipschitz cluster points. These cluster points will be 
shown to be fluid model solutions under the maximum pressure policy. Since 
the state space collapse result holds for the fluid model under the maximum 
pressure policy by Theorem 6, the network processes X^'™ asymptotically 
have the state space collapse. The latter result will be translated into the 
state space collapse result for diffusion-scaled processes X'' as r approaches 
infinity. 

Note that in Theorem 6 the state space collapse of the fluid model does 
not happen instantaneously after time if the initial state does not exhibit 
a state space collapse. The fluid-scaled processes X'"'"^ start from time rm in 
the original network processes. Hence, for m > 1, X''''" do not automatically 
have the state space collapse at the initial point. Such collapse can only be 
expected to occur in the interval [ro,L]. However, for m = 0, the state space 
collapse happens at time because of the initial condition (4.10), therefore, 
the collapse remains in the whole interval [0, L]. For this reason, we separate 
the proof into two parts according to the two intervals in the diffusion-scaled 
time: [0, L^r,o/'"^] and [roCr,o/'"^ > ^] • Note that the two intervals overlap be- 
cause of our assumption on the choice of L; if To^rfi/f'^ > T, the second 
interval disappears and the first interval covers the whole interval [0, T]. 

Propositions 2-5 develop a version of state space collapse for the interval 
[ro^rfi/'''^ ^T]. Proposition 2 shows that, on , the scaled processes X'''"^(-) 
are uniformly close to Lipschitz cluster points for large r. Proposition 3 shows 
that the above cluster points are solutions to the fluid model equations. In 
Proposition 4 we use Propositions 2 and 3 and Theorem 6 to prove the state 
space collapse of the fluid scaled processes X'''"^(-) in the time interval [tq, L] 
on the "good" set G'^ . The result in Proposition 4 is then translated into 



32 



J. G. DAI AND W. LIN 



diffusion scaling in Proposition 5 to give a version of state space collapse for 
the diffusion-scaled process X''(t) in the interval [To^rfl/f'^^T]. 

In Propositions 6-8, we show that the state space collapse occurs for the 
diffusion-scaled process W'{t) in the interval [0,L^r,o/'^^]- The basic idea is 
the same as the one described in the preceding paragraph except that now we 
only consider the scaled processes with m = 0. The corresponding network 
processes start from time 0, and by assuming the state space collapse hap- 
pens at time in Assumption 4, we have a stronger result for these types of 
processes: the state space collapse holds during the whole time interval [0,L] 
instead of just in [ro,-L]. In fact, the scaled processes X'''''(-) are proved to 
be uniformly close to some cluster points for which the state space collapse 
starts at time 0. These facts are stated in Propositions 6 and 7. In Propo- 
sition 8 we summarize the state space collapse result for the fluid-scaled 
process X'^''''(-) on [0,L] and translate it into a result for the diffusion-scaled 
process in [0, L^^.o/^^]- 

The results to be obtained in Propositions 5 and 8 are actually a mul- 
tiplicative state space collapse, as called in Bramson (1998). To obtain the 
state space collapse result stated in Theorem 5, we will prove that i^r,ml'^ 
are stochastically bounded at the end of this section. 

7.1. Probability estimates. In this section we give probability estimates 
on the service processes S'J'™(-), the routing processes $^'^'™'(-) and the upper 
bound of the fluctuation of the stochastic network processes X^'"'(-). 

Proposition 1. Consider a sequence of stochastic processing networks 
where the moment assumption, Assumption 3, is assumed. Fix e> 0, L and 
T. Then, for large enough r, 

(7.1) ^(^n.^^^\\S]^^{T]^ 

for each j £ J , 

(7.2) P(max ||cI.f'-(5;''"(T;'™(t)))-/f/.,^(7^^^^ 

for each j £ J and i £ Bj, 

(7.3) f( sup |X'^'™(t2) -X^'"^(ti)|>Af|i2- til + e /or some m<rr) 

where N = J{1 + \R\). 

Recall that = maxjj Rij. The following lemma is essential to the proof 
of Proposition 1. 
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Lemma 6. Assume that the moment assumption, Assumption 3, holds. 
Then for given T, and each j G J, 

(7.4) u^j'^'^^^/r -^0 as r ^ CO with probability 1, 

where u'-'^'™^^ = max{uj(£):l <l< Sj{r^T) + 1}. Furthermore, for any 
given e > 0, 

(7.5) F{\\<^{{l)-P^k\\n>en)<€/n 

for each j £ and i £ Bj, and large enough n, 

and for large enough t, 

(7.6) ^{\\S]{t)- i/jT\\t>et)<€/t for all j £ J, and all r. 

We delay the proof of Lemma 6 to Appendix B, and we now prove Propo- 
sition 1. 

Proof of Proposition 1. The proof here essentially follows the same 
reasoning as in Propositions 5.1 and 5.2 of Bramson (1998). We first inves- 
tigate the processes with index m, and then multiply the error bounds by 
the number of processes in each case, [rT] . We start with (7.1). 

Fix a J G ^7, and pick any ei G (0, 1] . Let r be the time that the first activ- 
ity j service completion occurs after time rm. We consider two cases. First 
suppose that r > rm + ^r,mTp"^{t). Then ^r,mTj''^{t) < t — rm < m^'^'™*^^/^^ 
and Sj{rm + Cr,mTj'"^ {t)) = Sj{rm). Thus, 

||5J(rm + er,™7j'''"(t)) - 5;(rm) - fi'j&;n.T;'"'{t)h < uf^^. 

Or if T < rm + S,r,mTj'"^{t), then by restarting the process 5J at time r, we 
have, by (7.6), that for large enough r, 

P(||5; (rm + ^r,mT;^"'{t)) - 5}(r) - f^{rm + ^r.mTj^'^it) - r)||^ > eiL^r,.,) 

<ei/{Lr)- 

we use the fact that Tj'"^{t) < t and S,r,m > r. Because <S'J(r) = Sj{rm) + 1 

J ^ r,T,max / r i 

and r — rm < u- j ix^, we have 
mS](rm^ir,mT]'^it)) 

- Sj{rm) - f/j(.r,7nTj'"'{t)\\^> |1 - U^j'^'^'^^'l + eiL^r,m.) 

<ei/(Lr). 

In both cases we have, for large enough r, 

P(||5;'™(77'™(t)) - ^,^77'"'(t)||^ > (1 +nf '"^^^)/e.,™, + 6iL) 

<ei/(Lr). 
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From (7.4), we have, for large enough r, 

(7.8) P(n^''^''""7r>ei)<ei. 

Let M^' denote the complement of the events in (7.8). Then, for large enough 
r, 

(7.9) FiM"^) > 1 - ei. 
Then we have, for large enough r, 

msriTrit)) - >{L + 2)6i) n Ml 

(7.10) < p[(i|5;''"(r;''"(t)) - /i,"r;''"(t)iiL 

>(l + uf''""^)/e.,™ + eiL)nA^n. 

In (7.10) we let r be large enough so that l/£,r,m < ei and, hence, (1 + 
Uj' '™^^)/^r,m < 2ei for all events in . 
It then follows from (7.7) and (7.10) that 

(7.11) P[(||5}''"(i;'"(t)) -/i5i;^'"(t)||L > (L + 2)ei)nM'-] < e^/{Lr). 
Inequality (7.11) holds for m = 1, . . . , \rT~\ . It then follows that 

max ||5;'"^(r,^'"^(t)) - f^^,T;'^{t)U >{L + 2)ei) n 
< \rT]ei/{Lr) <eiT. 
Then we have 

m^ax ||5;'™(l7''"(t)) - f.^^T;'-{t)U >iL + 2)e, 
<eir+(l-P(A^"))<(r+l)ei. 

Since ei can be chosen arbitrarily, we let ei = 1 A e/((L + 2) V (T+ 1)). Then 
(7.1) follows. 

Let 11^^^ be an upper bound on Then from (7.11), for large enough 

r, 

P[(S';'"'(T;'"^(L)) > (/i'^^^L + L + 2)) n M'] < €i/{Lr). 
From (7.5), replacing n by {fi^^^L + L + 2)^^^^, we have for each i £ Bj, 

p[(ll$f '"^(5;''"(r;'™(i))) - if 5}'"^(r;''"(t))iii 

(7.12) >ei(/i"^''^L + L + 2))nA^''] 
<ei/((^^"^^^L + L + 2)r) + ei/(Lr). 
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Let P™^^ = maxj(zj^i^i3. . Then from (7.11), we have 

P[(||if 5^'™(r[''"(t)) - J^'/i^T[''"(t)||i > P"^^^(L + 2)ei) nA^n 

(7.13) 

<ei/(Lr). 
It follows from (7.12) and (7.13) that 

> ei(P'^^" + ^u'"^" + 1)(L + 2)) n M'] 
< 5ei/(2Lr). 

This inequahty holds for m = l,...,[rT], so setting ei = 1 A e/ 
[((pmax _^ ^max _^ _^ 2)) V (ST + 1)], one gets 



( max '"'(5;'"^(r;''"(t))) - P^fJ^iTj'^mL > e) n 

\m<rT J J J J J 



<5Te 



1- 



Thus, (7.2) follows. 

Now we are going to show (7.3). First, it is easy to see that, for each j ^ J 
and each r, 

rj(i) -rj(s) <t-s forO<s<t 

along any sample path. Therefore, the bounds in (7.3) on components Tj 
are obtained with = 1. For components Zi,i G X, scaling (5.2) and apply- 
ing (7.1) and (7.2) gives that for any €2 > and large enough r, 

A sup \zr{t2)-zr{t^\ 

\0<ti<t2<L 

(7.14) > ^ (K^.|)|r;'"^(t2) - r;''"(ti)| + 2J(I + 2)62 for some m < rx] 
<2J(I + 2)e2. 

Since i?'' — > i? as r — > 00, we can choose r large enough so that \ W\ < + 1. 
Then, setting e2 = e/(2J(I + 2)) in (7.14) and iV = J(l + |P|), one gets (7.3). 
□ 



Let rmin > be the minimum r such that (7.1)-(7.3) are satisfied for some 
e > 0. Now for each r > rmin, we let e(r) be the smallest e such that (7.1)- 
(7.3) are satisfied. By Proposition 1, it is easy to see that 

lim e(r) = 0. 

1 VOO 

With e(r) replacing e, we call each of the events in (7.1)^(7.3) a "bad" event. 
We let denote the intersection of the complements of these "bad" events, 
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and the events in are referred to as "good" events in the rest of this 
section. Obviously, 

Urn P(gM = 1. 

r—>oo 

7.2. State space collapse on l^rflTo/r'^ ,T]. Again, let L > IVtq and T > 
be fixed. We divide the diffusion-scaled time interval [0, T] into two overlap- 
ping intervals: [0,^^,0-^/'^^] and [^r,oTo/r'^ ,T]. In this section we show a state 
space collapse result on the time interval [(,r,oTo/r'^,T]. The state space col- 
lapse on [0,^^,0-^/'^^] will be presented in the next section. 

In order to connect the fluid-scaled processes with the fluid model, we first 
introduce the notion of cluster point. Let F = B'^[0,L] be the space of right 
continuous functions with left limits from [0,L] to M^"'"'^, where d = I-|- J. Let 
C = {Fr} be a sequence of subsets of F. A point / in F is said to be a cluster 
point of C if for each e > and rg > 0, there exist r > rg and g G Fr such 
that f — Q]^ < e. The sequence {Fj-} is said to be asymptotically Lipschitz if 
there exist a constant k> and a sequence of positive numbers {€{r)} with 
e(r) ^ as r ^ oo such that for each r all elements f £ Fr satisfy both 

(7.15) 1/(0)1 < 1 
and 

(7.16) \f{t2)- fiti)\<K\t2-ti\+e{r) for ah ti, G [0, L]. 
Let F' denote those f £ F satisfying both (7.15) and 

(7.17) \f{t2)-f{ti)\<K\t2-ti\ foraUti,t2G[0,L]. 

The following lemma is due to Bramson (1998). We state it here for com- 
pleteness. 

Lemma 7 [Bramson (1998), Proposition 4.1]. Assume that C is asymp- 
totically Lipschitz. For each e > 0, there exists an rg, so that for each r > tq 
and g £ Fr, one has \\f — g\\L < e for some cluster point f of C with f S F' . 

Lemma 7 says that the cluster points are "rich": for large r, all elements 
in Fr can be uniformly approximated by cluster points. 
We set 

pr ^ {x^''"(.,cj), m<rT,oj£ G''} for each r 

and 



(7.18) 
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From the choice of our fluid scale S,r,m and the definition of T^'"^{-), it follows 
that 



It follows from (7.3) in Proposition 1 that the sequence of sets of scaled 
stochastic processing network processes X'''™(-) is asymptotically Lipschitz. 
Lemma 7 immediately implies the following proposition which says that, 
for large r, X'''™'(-) are uniformly close to cluster points that are Lipschitz 
continuous. 

Proposition 2. Fix e> 0, L and T, and choose r large enough. Then, 
for uj G Q''' and any m < rT, 



The next proposition says that if the stochastic processing networks oper- 
ate under a maximum pressure policy, then each cluster point of J-g satisfies 
fluid model equations (6.1)-(6.6), and thus is a fluid model solution to the 
fluid model operating under the maximum pressure policy. 

Proposition 3. Consider a sequence of stochastic processing networks 
operating under a maximum pressure policy. Assume Assumptions 1 and 3 
and that the limit network satisfies the EAA condition. Fix L > and T > 0. 
Then all cluster points ofj-g are solutions to the fluid model equations (6.1)- 
(6.6) on [0,-L], and they are all in F' . 

Proof. The idea is to approximate each cluster point of J-g with some 
^r,m. gr ghow that the equations (6.1)-(6.6) are asymptotically satis- 
fied by X'''™. We will only demonstrate equations (6.1) and (6.6); the others 
are quite straightforward and can be verified similarly. Let X = {Z,T) be a 
cluster point. We first verify equation (6.1). For any e > and all r that are 
large enough, there are uj gQ"^ and m < rT such that 



X'''™(0)| <1. 




(7.19) 
(7.20) 



\\sriTr(t))-^'jTrmL<e 

(5;'"^(T,^'"^(t)))-/fA.5i7'"^(t)llL<6 



for each j G J 



for each j £ J,i 



(7.21) 
(7.22) 
(7.23) 



Z(.)-Z^'™(-)||L<e, 
T(.)-r^'™(.)||^<e, 



W -R\< e. 
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Scaling (5.2) and plugging in the bounds in (7.19) and (7.21), we have 
|Zn"^(t) _ Z'-''"(0) +i?"r"'™(i)| < 2e for all t G [0,L]. 

From (7.22) and (7.23), we have, for each tG [0,L], 

\R^T^'^{t) - Rf{t)\ < \R'^{r-''^{t)-f{t))\ + \{R'- - R)f{t)\ 
< NJe + eNLJ. 

Recall that N > sup^ \ R^'\- It then follows that, for each t < L, 

\Z{t) - Z{0) + Rf{t)\ < \Z{t) - Z'-'^^it)] + |Z'-''^(0) - Z{0)\ 

+ \R''r'''^{t)-Rf{t)\ 

+ \Z'''"\t) - Z'-^'"\^) + KT-^''\t)\ 

< {4 + NLJ + NJ)e. 

Thus, equation (6.1) is satisfied by X because e can be arbitrarily small. 
To show equation (6.6), first observe that, for any allocation a£ A, 

\paia, Zit)) - p^(a, Z'^'™(t))| = |a X Z{t) ■ Ra - a x Z'"''"(t) • i?^a| 

< \a\{\Z{t) ■ {Ra-R''a)\ 

(7.24) +|(Z(t)-Z^'™(t))-i?^a|) 

< |a|((A^L + l)IJe + eIJA^) 

= {NL + N + l)\a\Ue. 

In (7.24), p^(^{a, q) = {ax q) ■ R^a is the network pressure for the rth network, 
associated with the parameter a, under allocation a when the queue length 
is q. 

Denote £* = argmaxag£-pc»(a, Z(t)) as the set of maximum extreme allo- 
cations under buffer size Z[t). Because the limit network satisfies the EAA 
condition, we can choose an a* G 8* such that Zi[t) > for each constituent 
buffer i of a*. Denote I (a*) the set of constituent buffers. Namely, 




Then Zi{t) > for all i G I (a*). 

Suppose a G \ <?*. Then pa{a, Z{t)) < maxa'es Pa{a' , Z{t)). Since 

Pa{a*,Z{t)) =maxpa{a',Z(t)) 

and minjgj(„*) Zj(t) > 0, by the continuity of X(-), there exist ei > and 
5 > such that, for each r G [t — 5,t + 6] and i G X(a*), 

(7.25) Pc,ia,Z{T))+ei<Pc,ia*,Z{r)) and Zi(r)>ei. 
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For sufficiently large r, we can choose e small enough such that {NL + 
iV + 1) X |a|IJe < ei/3. It follows from (7.21), (7.24) and (7.25) that, for ah 
r G [t - (5, t + (5] , 

p;(a, Z^'-(t)) + ei/3 < p^(a*, Z'^'-(r)), 

Z[''"(r) > ei/2 for each i G J(a*). 

Choosing r > 2J/ei , for each r G [rm + ^r,m(i — i^) i '''"i + S,r,m(t + <J)] > we have 

(7.26) p;(a,Z^^(T))</Ja*,Z^Xr)), 

(7.27) Z[(t) > J for each i G T(a*). 

Condition (7.27) implies that a* is a feasible allocation at any time r G [rm + 
&,m{t — S),rm + ^r,mit + S)], that is, a* G "^(t). Following (7.26) and the def- 
inition of a (preemptive-resume) maximum pressure policy, the allocation a 
will not be employed during time interval [rm + ir,m{t — ^) , rm + (,r,m{t + S)] . 
Therefore, only the allocations in £* will be employed during this interval. 

For each a € £, denote {T°'Y{t) to be the cumulative amount of time 
allocation a has been employed by time t. Because only allocations in 8 are 
employed under a maximum pressure policy, we have that, for each r and 
all t > 0, under a maximum pressure policy, 



Furthermore, since the employment of allocation a for one unit of time 
contributes to Oj unit of activity j processing time, we have 



T^{t) = Y^a{T-Y{t). 



Then it follows that 



(a X Z{t)) ■ {R[T^rm + Cr,m{t + ^)) " ^'X™ + Cr,m{t - 





Y^iia X Z{t)) ■ Ra)[{Ty{rm + ir,m{t + ^)) 



-{T-Y{rm + irAt-5))] 



J2 ((« X Z{t)) ■ Ra)[{T''Y{rm + ^r,m{t + 5)) 



40 J. G. DAI AND W. LIN 

(7.28) 

-{Ty{Tm + irAt-m 

= fmax(a x Z{t)) ■ Ra] V [(T")"(rm + ^r,m{t + 6)) 
^^^^ Kl^* 

-{T''Y{rm + ^r,m{t-m 
= (max{a x Z{t)) ■ Ra) Y [{T'^Y {rm + ^r,mit + 6)) 

-(T''Y{rm + ^r,mit-m 

= m^i max(a x Z{t)) ■ Ra ) . 
' \ ae€ J 

The second and fourth equahties in (7.28) hold because only allocations in 
£* will be employed during [rm + (,r,m{t — 6),rm + ^r,m{t + (5)]; the third 
holds because every allocation a ^ £* has the same network pressure equal 
to maxag£:(a x Z{t)) ■ Ra. From (7.28), we have 

(a X Z{t)) ■ R{T''^"'{t + 6)- T''^"'{t - 5))/25 = max(a x Z{t)) ■ Ra. 

Because e in (7.22) can be arbitrarily small, we have 

(a X Z{t)) ■ R{f{t + 6)- f{t - 6))/25 = max(Q x Z{t)) ■ Ra, 

and by letting 6 — > oo, (6.6) is verified. 

Since X and X*"'™" are uniformly close and X**'™ satisfies (7.15) and (7.16), 
it is straightforward to verify that X satisfies both (7.15) and (7.17), thus 
showing XeF'. □ 



Since every fluid model solution under the maximum pressure policy sat- 
isfies (6.7) by Theorem 6, Proposition 3 implies that any cluster point X of 
J^g satisfies (6.7). That is, 

\Z{t)-CW{t)\=0 fort>ro, 
where W{t) = y* ■ Z{t). Define 

Because the fiuid-scaled stochastic processing network processes can be uni- 
formly approximated by cluster points, it leads to the following proposition. 

Proposition 4. Fix L > tq, T > and e > 0. For r large enough, 
\Z''^^{t)-CW'^'^{t)\<e for all < m < rT,To < t < L,LO e G"^ . 
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Proof. From Proposition 2, for each e > 0, large enough r, and each 
< m < rT and oo ^Q''\ we can find a cluster point X such that 

\\Z{.)-Z^^^\.)\\^<e. 

From Proposition 3 and Theorem 6, we have 

\Z{t)-CW{t)\=Q ioTt>To. 

Then, for each t > tq, 

(7.29) |Z"'™(t) - ("^^"''"(t)! < |Z'^'"'(t) - Z{t)\ + |C"T^(t) - ("^"'""(i)!- 
Because ^ y* as r ^ oo, for large enough r, 

Let K = {supj. I y'' I ) V I I < oo . Then we have 

\W{t) - W^'"'it)\ < \y* ■ Z{t) - / • Z{t)\ + \f . Zit) - / • Z'''™(t)| 

< I(e|Z(t)| + ne) < {NL + 1 + K)Ie. 

The last inequality follows from the fact that Z{t) < NL + 1 for all t < L. 
Note that Z{t) G F' and thus satisfies (7.17). One also gets W{t) < k{NL + 1) 
since \y*\ < k. From (7.29), we have 

_ C"VF'-''^(t)| < e + k[NL + 1 + K)Ie. 

The proposition follows by rechoosing e. □ 

We need to translate the results in Proposition 4 into the state space 
collapse results for diffusion-scaled processes. First we can express Z^'"^(t) 
by via 



where ir,m = ir,m/i'- The time interval [ro,L] for the fluid-scaled process 
2;r,m corresponds to the time interval [{m + S^r^rnTo) /f, {m + S,r,mL)/r\ for the 
diffusion-scaled process Z^' . Proposition 4 immediately leads to the following. 

Proposition 5. Fix L> tq, T > and e > 0. For r large enough and 
each m < rT, 

\Z^{t)-CW''{t)\<ir,me 

(7.30) 

for all {m + (,r,mTo) / r <t<{m + ^r,mL)/r, uj^Q'' . 



42 



J. G. DAI AND W. LIN 



Proposition 5 gives estimates on each small interval for \Z^(t) — (°'W^'{t)\. 
We shall obtain the estimate on the whole time interval [0,T], and then 
show that ^r,m are stochastically bounded. The following lemma ensures 
that for large enough L, in particular, for L > 3Ntq + 1, the small intervals 
in Proposition 5 are overlapping, and therefore, the estimate on [S.rflTo/r,T] 
is obtained. 

Lemma 8. For a fixed T > and large enough r, 

for uj G Q''' and m < rT , where N is chosen as in Proposition 1. 
Proof. By the definition of Q"^ , 

|Z'''™(t2)-^''''"(tl)| <iV|t2-tl| + l 

for ti,t2 £ [0,-^^] and m < rT. Setting ti = and t2 = l/(,r,m, we have 

(7.31) |Z^((m + l)/r)| - (m/r) | < + ^r,m. 
From the definition of (,r,m+i, we have 

er,m+i = |^"((m+l)/r)| VI 

<{\Z^{m/r)\+N + ^r,m)Vl 

The first inequality follows from (7.31), and the second inequality follows 
from the definition of N and ^r,m- D 

7.3. State space collapse on [0,^^,0-^^/^^]- Now we shall estimate \Z'^{t) — 
("W'^{t)\ on the interval [0,^^,0-^/?^] = [0,^r,oL/r'^]. This wih be given by the 
initial condition (4.10) and the result in the second part of Theorem 6. 

Condition (4.10) implies that 

Z'''°(0) ^0 in probability. 
Then, for each r > 0, we let 

(7.32) ei(r) = min{e : P(Z^'°(0) > e) < e}. 
It follows from (7.32) that 

lim ei(r) — > 0. 

) — >oo 

Now let be the intersection of Q^' and the event 

^'■'°(0)<ei(r). 
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Obviously, lim^^oo IP(>CO = 1. 
We define 

with 

F:; = {X''^{;co),u;eC}. 

Parallel to Proposition 2, we have the following proposition which states 
that To can be asymptotically approximated by cluster points of J^o- 

Proposition 6. Fix e > 0, L > and T > 0, and choose r large enough. 
Then, for to £ , 

||X-'0(.,u;)-i(-)||L<e 
for some cluster point X(-) of J-'o with X(-) € F'. 

Proof. Since both (7.15) and (7.16) are satisfied by T"'^, the result 
follows from Lemma 7. □ 

Proposition 7. Fix L > 0. Then for any cluster point X(-) of J^o, 
Z{t) = CW{t) forte[0,L]. 

Proof. Since any cluster point of J^o is automatically a cluster point 
of J^g, therefore, it satisfies all the fluid model equations. It suffices to show 
that 

(7.33) \Z{0)-CW{0)\=0, 

which, together with Theorem 6, implies the result. In fact, we have Z{0) = 0, 
since for any given 5 > 0, one can choose r large enough and uj £ such 
that |Z(0) - ^'■'0(0) I < 6, and Z^'O(O) <5. □ 

Propositions 6 and 7 immediately lead to the following proposition, which 
is parallel to Propositions 4 and 5. 

Proposition 8. Fix L and e > 0. For large enough r, 

\Z'''^{t) - C"VF'^'°(i)| < e for alio <t<L,uj€C 

and 

(7.34) iZ^t) -CW'^{t)\<^r,oe for allO<t<^r,oL/r,uj£r. 
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7.4. Proof of Theorem 5. Propositions 5 and 8 together give the multi- 
plicative state space collapse of the stochastic processing network processes. 
To prove the state space collapse result stated in Theorem 5, it is enough 
to prove that ^r,m 

are stochastically bounded. We first give an upper bound 
on ^r,m in terms of W^. 

Lemma 9. // \Z^'{m/r) — (^°'W'^{m/r)\ < 1, there exists some k>1 such 
that 

&;m<<W'{m/r)Vl). 

Proof. Because ^r,m = \Z^{rm)\ Vr, we have 

Cr,m = \Z\mlr)\ V 1 < \CWirnlr) + 1 < 2(|C"|W^''(m/r) V 1). 
The result then follows by choosing ft; = 2(|(^°|Vl). □ 

The following proposition will be used to derive an upper bound on the 
oscillation of W"^ . 

Proposition 9. Consider a sequence of stochastic processing networks 
operating under the maximum pressure policy with parameter a . There exists 
eo > such that, for large enough r, and any <ti < t2, ifW^'{t) > J/(reo) 
and \Z''{t)/W''it) -C\<eo for all t £ [ti.ta], then 

Y^{t2) = Y^{ti). 

Proof. First, we can choose < eo < minjCf /2 : Cf > 0} so that for all 
i with > and all t E [ti,t2], we have 

Z:it) > W^mC - eo) > W^it)eo > J/r. 
That is, for all i with Cf > 0, 

(7.35) ZI{t)>3 for all r G [r^t i , r^ts] . 

Define 

£* = argmaxy* • Ra = argmax(a x (") ■ Ra. 

The second equality follows from the fact that a x C"' = y* / Ylii^iiiVi)'^ / (^i) ■ 
We now show that at least one of the allocations in £■* is feasible dur- 
ing [r^ti,r^t2]- Define T{a) = {i £ T :J2j Bjiaj > 0} to be the constituency 
buffers of the allocation a. One sufficient condition for an allocation a to be 
feasible at time t is that Z^t) > J for all i G I (a). Because the limit network 
satisfies the EAA condition, replacing q by y* in Definition 4 implies that 
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there exists an allocation a* G £* such that y| > for each i G X(a*); namely, 
all constituency buffers of a* have positive y^'s. Since y* > implies > 0, 
it follows from (7.35) that Z[(t) > J for all r e [r2ti,r-2f2] and iGl{a*). 
This implies that a* is a feasible allocation during [r^ti,r^t2]- 

Next, we will show that if eg is chosen sufficiently small, only allocations 
in £* can be employed under a maximum pressure policy during [r^ti,r^t2]- 
Let 



Set 



ei= [y* ■ Ra* — max y* ■ Ra ] > 0. 

ae£\£* 



Ko = supmax |i?'"a| and ki = y^((y*)^/ai). 

Choose r large enough such that — < eg. Then for each alloca- 

tion a^E\E* and each r G [r^t i , r^t2] , 

((a X Z"(r)) • Wa* - (ax Z'"(r)) • i2'"a)/VF"(r) 
'a X Z^(r) 



a X C° • Wa* 
+ {y* ■ R^a* — y* ■ Ra* ) /ki + (y* • Ra* — y* ■ Ra) / ki 

(7.36) 

+ {y* -Ra-y*- R'a)/K^ + {a x C - "^f^^ ) • i?^a 

> — |a|eoKoI — eoJ/^i + ei/^i — eoJ/^i — |a|eoKoI 
= (ei - 2{\a\KQtiil + J)eo)/Ki. 

Thus, 

p^(a*, Z'^(t)) - p^(a, Z^(r)) > t^^(T)(ei - 2(|a| a^o^iI + J)eo)M. 
Set eo = ei/(3(|a|)KoKiI + J)) Aminigj{Cf/2:Cf >0}, then 
(7.37) f^{a*,Z^\r))-p^^{a,Z\T))>Q. 

Obviously, eo depends only on a, R and A. Equation (7.37) implies that the 
pressure under allocation a* is strictly larger than that under allocation a. 
It follows that only the allocations in £* can be employed during [r^ti,r^t2]- 
Therefore, for large enough r, 

a€£ 

(7.38) 
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= ^(l-/)((TT(r't2)-(TT(^'ti)) 

where the third equahty follows from Lemma 10 below. Equations (7.38) 
and (5.1) imply that Y''{t2) - Y''{ti) = 0. □ 

Lemma 10. For each a* £ £* = argmaxag^y* • Ra and large enough r, 
■ Rl'a* = maxy'' • RJ'a = 1 — . 

The proof of Lemma 10 will be provided in Appendix B. We now complete 
the proof of Theorem 5. 

Proof of Theorem 5. We only need to show that ^r,m are stochasti- 
cally bounded. We first fix an e > 0. Because VP^^(O) is stochastically bounded, 
there exists a ki such that, for r large enough, 

P(VF''(0) < Ki) > 1 - e. 

Recall that the process X*, defined in Lemma 3, is a Brownian motion, 
so it has continuous sample path almost surely. Therefore, there exists a K2 
such that 

P(Osc(X*, [0,r + L\) < K2/2) > 1 - e/2, 

where 

Osc(/,[0,t])= sup /|(t2)-/(ti)|. 

0<ti<t2<t 

Since X^' converges to X* in distribution, for large enough r, 
P(Osc(X", [0,r + L]) < K2) > 1 - e. 
Meanwhile, because 

|Z^(0) - CW''{0)\ ^0 in probability, 

we have 

¥{\Z^O)-CW'{0)\<e)>l-e 

for r large enough. 
Define 

TC'' = {uj : W''{0) < Ki,Osc{X'', [0, T + L]) < K2, 
and \Z''{0)-CW''{0)\<€}. 
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Then for r large enough, 

P(W'') > 1 - 3e. 

Furthermore, we can choose r large enough such that Propositions 5 and 8 
hold with e replaced by e/ k{ki + K2 + 1) and P(£'") > 1 — e, where k is given 
as in Lemma 9. Note that T{C') ^ 1 as r ^ oo. 

Denote AA*"'*^ = C n TC''^, then for all r large enough, 

¥{N''^') > 1 - 4e. 

Now we are going to show that if e < eo, Cr,m < k,{ki + K2 + 1) on M^'"^ for 
all r large enough and m < rT. In fact, we are going to show the following 
is true on M^''' for all r large enough and m < rT: 

(7.39) \Z''{m/r)-CW'^{m/r)\<e, 

(7.40) ^r,m< At(Ki + K2 + l), 

(7.41) \Z''{t)-CW''{t)\<e for all 0<t< (m + er,m^)/r, 

r(m+^r,mL)/r ^ 

(7-42) I l(^.(^)^,)dy^(.) = 0. 

This will be shown by induction. When m = 0, (7.39) obviously holds on 
AT'-'S and 

&,0 < k{W''{0) V 1) < k{ki V 1) < k{ki +K2 + 1). 

Meanwhile, from (7.34), replacing e by €/k{ki + K2 + 1), we have 

\Z''{t) - CW''it)\ < e for all t G [0,^r,oL/r]. 
Then from Proposition 9, with r > 2J/eo, we have 

Now we assume that (7.39)-(7.42) hold up to m, and we shall show they 
also hold for m + 1. First, (7.41) for m directly implies (7.39) for m + 1 
because £,r,mL > 1. Next, because (5.7) and (7.42) hold, and by Theorem 5.1 
of Williams (1998b), we have 

OsciW"-, [0, (m + Cr,mL)/r]) < Osc(l^ [0, (m + Cr,r,^L)/r]) + 1 < ,^2 + 1. 

Note that, because m < rT and (7.40), we have (m + ^r,mL)/r <T + L for 
r large enough. It then follows that 

ir,m+i<ti{W''{{m+l)/r)\J\) 

< k(I?^(0) + Osc{W'', [0, (m + l)/r]) V 1) 

< KiW'iO) + Osc{W', [0, (m + Cr,m.L)/r]) V 1) 

< k{ki + K2 + 1). 
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This fact and (7.30) imply that 

\Z'\t)-CW''{t)\<e 

for ah t G [(m + 1 + Cr,m+iTo)/r, (m + 1 + ^r,m+iL)/r]. 

We choose L > 3Ntq + 1. Then Lemma 8 impHes ^r,mL/r > (1 + ^r,m.+i'^o) /f- 
It follows that 

- CW''{t)\ < e for ah t G [0, (m + 1 + Cr,m+i^)/r]. 

Then again Proposition 9 gives 

Therefore, we can conclude that ^,.,m < k{ki + K2 + 1) for all < m < rT, 
which implies 

\\Z'-{t)-CW'{t)\\T<e onAf''' 

for all large enough r. Theorem 5 follows because e can be chosen arbitrarily 
small. □ 

8. Heavy traffic limit theorem. In this section we prove the heavy traffic 
limit theorem, Theorem 4. Our proof employs an invariance principle devel- 
oped in Williams (1998b) for semimartingale reflecting Brownian motions 
(RBMs), specialized to the one-dimensional case. 

Recall that the constant eo is defined in Proposition 9. For each r, define 
process 6^' = {S'^{t) :t > 0} as 

6^{t) = {\Z^{t)-CW^{t)\V{2J/r))/eo. 

It follows from the state space collapse theorem, Theorem 5, that 5^ — > in 
probability as r — > oo. Now, for each r, define processes Y' = {j^it) :t > 0} 
and W'' = {W'^{t):t>0} as 

Y\t)=W'^{t)A6'^{t) and W''{t)=W''{t)-Y{t). 

Since 5'' — > in probability as r — > oo , 7*^ ^ in probability as r ^ 00 . It is 
easy to see that W^'it) = iW''{t) - (5'^(t)) V > for ah t > 0. Now we claim 
that 

(8.1) / W''is)dY'\s) = 0. 

Jo 

To see this, it is enough to show that for any < ii < ^2, ^^(^2) = ^^(^i) 
whenever W'^{t) > for all t G [^1,^2]; see, for example. Lemma 3.1.2 of 
Dai and Williams (2003). Suppose that W''{t) > for all t G [ti,t2]- Then 
W''{t) > 6''{t) for ah t G [ti,t2]- The latter condition implies that W^'it) > 
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2J/(reo) and \Z'"{t)/W''{t)-C\ < eo for alH G [h,t2]. It follows from Propo- 
sition 9 that Y''{t2) = Y^'iti), thus proving (8.1). 

By Theorem 1, the maximum pressure policy is asymptotically efficient, 
and thus, by Lemma 3, X'' converges in distribution to the Brownian motion 
X* as r — > oo . 

Therefore, we conclude that the processes (W^ jX"^ ,Y^) satisfy the follow- 
ing conditions: 

(i) W' = X'' + Y\ 

(ii) W = W'' + Y, where W^'it) > for all t > 0, and 7^ ^ in proba- 
bility as r — > 00 , 

(iii) X^ converges to the Brownian motion X* in distribution as r ^ 00. 

(iv) With probability 1, 

(a) y'^(o) = o, 

(b) y is nondecreasing, 

(c) J,^W^{s)dY^{s) = 0. 

Since we have a one-dimensional case, condition (II) of Proposition 4.2 
of Williams (1998b) is satisfied, and consequently, condition (vi) of The- 
orem 4.1 of Williams (1998b) holds. It then follows from Theorem 4.1 of 
Williams (1998b) that converges in distribution to the reflecting Brow- 
nian motion W* = ^(X*). The convergence of to ^°'W* follows from 
the state space collapse result (5.16). Thus, we have completed the proof of 
Theorem 4. □ 

9. Linear holding cost. In this section we discuss the asymptotic opti- 
mality in terms of a linear holding cost structure for the sequence of stochas- 
tic processing networks. For each network in the sequence, we assume that 
there is a linear holding cost incurred at rate Cj > for each job in buffer 
i. Let c be the corresponding vector. For the rth network, the total holding 
cost rate at time t is 

(9.1) C"(t) = c-Z"(i). 

Define the diffusion-scaled linear holding cost rate processes of the rth 
network = {(5^(t),i > 0} via 

C''{t) = C'^{rh)/r. 

Clearly, (7^(t) = c • Z'^(t) for t > 0. 

Definition 7. Consider a sequence of stochastic processing networks 
indexed by r. For a given e > 0, an asymptotically efficient policy vr* is 



50 



J. G. DAI AND W. LIN 



said to be asymptotically e-optimal for the linear holding cost if for any 
t > 0, > 0, and any asymptotically efficient policy vr, 

(9.2) limsupP(C;.(t) >r?) <liminfP((l + e)(5;(t) >r?), 

r — >oo ^ 

where C^-*{t) and C^{t) are the diffusion-scaled total holding cost rates at 
time t under policies vr* and vr, respectively. 

Theorem 7. Consider a sequence of stochastic processing networks where 
Assumptions 1~4 hold and the limit network satisfies the EAA condition. For 
any given e > 0, there exists a maximum pressure policy vr* that is asymp- 
totically e-optimal for the linear holding cost. 

Proof. Firs, we note that under any policy, for all r, 

C-{t) = c-Z'{t)> (mmQ/y[) W\t). 

Let i G arg minjgx Cj /y* . Then for any asymptotically efficient policy vr, 

liminf P(C;(t) > r/) > liminf P((c,/yn^,?(*) > ^) > H{cJyl)W*{t) > r?). 

The second inequality follows from (5.10) and the fact that y''' — > y* as r — > 
oo. 

Now consider a maximum pressure policy vr* with parameter a given by 

' Ciy*e/{l\cxy*\), i = L, 
1, otherwise. 



Because Ei'iVi'f/ai' > {vtf/a, = y*I|c x y*\/c,£, we have qC" < c,e/{y*I) 
for all i^ L and qC" < cjy*. 
From Theorem 4, we have 



Because 



we have 



CI* c-Z*'" asr^oo. 



Y^CiZrit) = Y.c,CW*{t)<{cJy:)W*{t){l + e) 



lim P(C;, (t) >7])< nicMW*{t){l + e)> 7]). 

r — ^oo 

Then (9.2) follows. □ 

Theorem 7 says that, for any e > 0, one can always find a maximum 
pressure policy such that, for any time t, the total holding cost rate under 
the maximum pressure policy is asymptotically dominated by (1 -|-e) times 
the total holding cost rate under any other efficient policy in the sense of 
stochastic ordering. For a given e, the parameter a that is used to define the 
asymptotically e-optimal maximum pressure policy in Theorem 7 depends 
on the first-order network data R and A. 
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APPENDIX A: AN EQUIVALENT DUAL FORMULATION FOR THE 

STATIC PLANNING PROBLEM 

In this section we describe an equivalent dual formulation for the static 
planning problem (3.1)-(3.5). The equivalent dual formulation will be useful 
for our proofs in Appendix B. Throughout this section, we will consider an 
arbitrary stochastic processing network, so the results developed here can be 
applied to each of the rth network in the network sequence that we discussed 
in Section 4 and they can also be applied to the limit network. The main 
result of this section is the following proposition. 



Proposition A.l. Suppose p is the optimal objective value to the static 
planning problem (3.1)-(3.5). Then {y*,z*) is optimal to the dual prob- 
lem (3.6)-(3.10) if and only if y* satisfies 

(A.l) max ^ ylRijttj = -p 

and 

(A. 2) max ^ y^Rijaj = 1, 

and {z^, k G /C/} and {z^, k G /C5} are, respectively, optimal solutions to 
(A.3) min - ^ Zk 



(A.4) s.t. "^^ylRij < — ^ ZkAkj for each input activity j; 
and 

(A. 5) min ^ Zk 

(A. 6) s.t. ^^ylRijl^ ^ ZkAkj for each service activity 
(A. 7) > for each service processor. 



Proof. For the "only if" part, we assume {x*,p) and {y*,z*) are an 
optimal dual pair for the static planning problem and its dual problem. We 
will show that y* satisfies (A.1)-(A.2) and z* is optimal to (A.3)-(A.4) and 
(A.5)-(A.7). We first show that 

(A.8) max ^ y^Rijaj > -p, 
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and 

(A. 9) max V y*Rijaj > 1. 

For this, we construct a feasible allocation a* with 



Xj/p, j(^Js- 

By the complementary slackness on the constraints (3.7) and (3.8), we have 

(A.io) ylRij^*3 = - E 

and 

(A.ii) y*R^3Xj= E 4Akjx*. 

By the complementary slackness on the constraints (3.3) and (3.4), we have 
(A.12) Y z*,AkjX*=J2 4 = P 

j€Jl,k£Ki k€Ki 

and 

(A. 13) Y 4AkjX* = pY 4 = P- 

jeJsMlCs k&Ks 

The last equality in (A.12) is from the strong duality theorem; the optimal 
objective value of the dual problem equals the optimal objective value of the 
primal problem. Readers are referred to Section 4.2 of Luenberger (1984) 
for a formal description of the strong duality theorem. The last equality 
in (A. 13) follows from (3.9). Then from the definition of a* and (A.IO)- 
(A.13), it immediately follows that 

E yiRijO'*j= E y*RijX*j = -P 

and 

E yiRijo*j= E yiRijx*j/p = 'i-, 

which imply (A. 8) and (A. 9) because a* G A. 
Next we shall show that 

E yi^ijO-j < -p 
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and 



max 



For any a£ A, we have 



J2 Vi^ij^j ^ 1- 



■E4 

keKi 



The first inequahty above fohows from (3.7), and the nonnegativity of o; the 
second inequahty holds since aG A and, therefore, J2jejj ^kjO-j = 1 for each 
k E: K-i; the third is due to the strong duahty theorem. Similarly, we have 

Y ylRijo-j < Y 4Akjaj < Y. 4 = 1- 

The first inequality above follows from (3.8); the second follows from 
J2j£js ■^kjO'j < 1 and 2fc > for each k S K,s\ and the third is due to (3.9). 

Hence, y* satisfies (A.l) and (A. 2). To see the z* is an optimal solution 
to (A.3)-(A.4) and (A.5)-(A.7), we consider the following problems: 



(A.14) 



and 



(A.15) 



max 



s.t. 



max 



s.t. 



Y Akjaj = 1 

jeJi 

Uj > 0, 



Y Vi^ij^^j 



for each input processor k, 



Y^ ^kjO-j ^ 1 for each service processor fc, 

i&Js 



aj > 0, 



j G Js- 



It is easy to see that the above two problems are equivalent to the left- 
hand side of (A.l) and (A. 2). Furthermore, they are the dual problems of 
(A.3)-(A.4) and (A.5)-(A.7). This imphes that the optimal objective values 
to (A.3)-(A.4) and (A.5)-(A.7) are —p and 1 respectively. Because {y*,z*) 
is an optimal solution to (3.6)-(3.10), (A. 4) and (A. 6) are satisfied by z* , 
J2kGJCi 4 — Pi and X^fceACs -^^ = 1. This implies that z* is feasible to (A.3)- 
(A.4) and (A.5)-(A.7) with respective objective values —p and 1. Therefore, 
z* is optimal to (A.3)-(A.4) and (A.5)-(A.7). 

For the "if" part, let {y*,z*) be such that y* satisfies (A.l) and (A. 2) 
and z* is optimal to (A.3)-(A.4) and (A.5)-(A.7). Because (A.3)-(A.4) and 



54 J. G. DAI AND W. LIN 

(A.5)-(A.7) are dual problems of the equivalent formulation of the left-hand 
side of (A.l) and (A.2), 

(A.16) E4 = P 

and 

(A.17) E^^ = l- 

fce/Cs 

The fact that z* is feasible to (A.3)-(A.4) and (A.5)-(A.7), together with 
(A.17), implies that {y*,z*) is feasible to the dual problem (3.6)-(3.10). 
Furthermore, the corresponding objective value is p because of (A.16). This 
implies that {y*,z*) is optimal to (3.6)-(3.10). □ 

Proposition A.l immediately leads to the following corollary because the 
problem maXa^A U* " can be decomposed into problems (A. 14) and (A. 15). 

Corollary A.l. Suppose {y*,z*) is the unique optimal solution to the 
dual problem (3.6)-(3.10) with objective value p. Then y* is the unique I- 
dimensional vector that satisfies 

(A. 18) max y*Rijaj = 1 

and 

(A. 19) max y* ■ Ra = l- p. 

aGA 



APPENDIX B: PROOFS OF LEMMAS AND THEOREM 1 

In this section we provide the proofs for Lemmas 1, 2, 5, 6 and 10, and 
Theorem 1. 

Proof of Lemma 2. From Corollary A.l and Assumption 2, we have 

max y^ ■ Ra = I — p^ . 
aeA 

On the other hand, T'^[t)/t G A. Hence, [1 — p'')t > y^' ■ RT'' (t) , which implies 
y is nonnegative. Similarly, for any <ti <t2, {T^{t2) — T^(ti))/(t2 — ti) G 
A, and we have 

y^ ■ R{T^{t2) - T^{ti))/{t2 - ti) < (1 - p^. 
It follows that V{t2) - ^{ti) > 0. □ 
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Proof of Lemma 5. First, if the statement is not true, then we can 
find avo£V° such that kvq ^ ^ for ah < k < 1 because of the convexity of 
V. Denote Vq = {kvq, < k < 1}. Because any u in Vq is not inV, VoCiV = . 
It is easy to see that Vq is relatively open and convex. Therefore, there exists 
a hyperplane separating V and Vq [cf- Rudin (1991), Theorem 3.4]. In other 
words, there exists a vector y and a constant b such that y ■ v <b for all 
V £V and y ■ v > b ioi all v £ Vq. We notice that b must be zero. To see this, 
first we have 6 > because the origin is in V. Moreover, for any e > 0, we can 
choose K arbitrarily small such that Ky ■ vq < e. Because kvq S Vq, we have 
b < Ky ■ Vq < e. This implies 6 = 0, therefore, the origin is in the separating 
hyperplane and max^gyt y ■ Ra = 0. Obviously, yi^y* because y* - v = <y-v 
for V G Vq. Since maXa£AJ2iel,jeJs Vi^ij^j > vector y, we consider 

two cases: 

Case 1: maXa^AJlieijeJs Vi^ij^j > 0- ^^i^ case, without loss of gener- 
ality, we select y such that maXaeAJ2i£i,j£jsyi^ij'^j — 1- Then y satis- 
fies both (A. 18) and (A. 19) with p= 1. On the other hand, from Corol- 
lary A.l, y* is the unique vector that satisfies both (A. 18) and (A. 19). 
This is a contradiction. 

Case 2: maxag^^ J2i£i,j£js yi^ij^j = 0- For this case, we will show that y* + 
y satisfies both (A. 18) and (A. 19) with p= 1, thus yielding a contradic- 
tion. For this, first observe that x* G argmaxag_4y* • Ra. This implies 
that 



so that y* ■ Ra > y* ■ Rx* . This is certainly not true. 

We also observe that x* E argmaxag_4 y ■ Ra because maxag^ y ■ Ra = 
and X* satisfies (3.2). This again implies that 



x*Gargmax y*Rijan. 




To see this, suppose there is an allocation a£ A such that 



then we can define another allocation a by 




X* G argmax 



Thus 



max(y* -|- y) ■ Ra = {y* + y) ■ Rx* = 

a£A 
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and 

•ieJjGJs i€l,j€Js 

Proof of Lemma 6. It is natural to work in a more general setting. 
Consider an i.i.d. sequence of nonnegative random variables {ve,i =1,2,...} 
with mean 1/ fiy. Assume V£ have finite 2 + moments for some Cy > 0. 
That is, there exists some a < oo such that E(f^''''^") = a. Let V{n) = 
X]"=i'^f)^ G Z^. Define the renewal process associated with V{n) as G{t) = 
max{n : V{n) < t}. Let ^j^-'^.^ax ^ rnax{u^ : 1 < ^ < ^(r^r) + 1}. 

It immediately follows from Lemma 3.3 of Iglehart and Whitt (1970) that 

(B.l) 7;''''^'™^^/r ^ with probability 1. 

We now show that for any fixed e > and large enough n, 

(B.2) mV{i)-i/^^v\\n>en)<e/n. 

Because vi have finite 2 + e„ moments, one gets 

E{\V{e) - £/Hv\^^'^) < Kyt^^-"!'^ for all l<n, 

where is some constant that depends just on a and e„ [cf. Ata and Kumar 
(2005), Lemma 8]. Then from Chebyshev's inequality, we have, for each 
l<n, 

^{\V{l)-lliiy\ > en) < Kj^+'-/y{enf+'- < Kj{e^+'-n^+'-/^). 
Choosing n large enough, 

F{\V{i)-i/^iy\>en)<e/n. 

Let 

T = mm{e:\V{i)-i/fiy\>ne}. 
Then, restarting the process at time r, 
F{\V{n) - n/fiy\ < en/2 | r < n) 

< P(|y(n) - V{t) - (n - T)/fiy\ > en/2 | r < n) < e/2n. 
On the other hand, 

F{\V{n) - n/fiy\ < en/2) > 1 - e/2n. 

It then follows that P(r < n) < e/(2n — e) < e/n. This implies the result. 
Finally, the inversion of (B.2) gives 

(B.3) IP(||G'(s) - iJ,vs\\t > et) < e/t for large enough t. 
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Readers are referred to Proposition 4.3 of Bramson (1998) for such an in- 
version. 

Applying (B.l) and (B.3) to the utilized service times Uj{i), one gets (7.4) 
and (7.6) in Lemma 6. Using (B.2) for each component of the routing vector 
(jyi{£) yields (7.5). □ 



Proof of Lemma 1. We first show that {{y^',z'')} is bounded. Since 

-2^ > 0, J2keK.s ■^k~^^ SfceA;/ ~ '^^ have < 1. To show 

{y^} is bounded, we consider the following primal-dual pair: 

(B.4) minimize p 

(B.5) subject to Rx > be, 

(B.6) E^kjx, 

(B.7) 

(B.8) 
and 

(B.9) maximize 



x>0, 



< 



for each input processor k, 



for each service processor k, 



k&Ki iei 



(B.IO) subject to 



(B.ll) 



y ] Zk^kj 
keKi 



keKs 

y > 0; and Zk>0 



for each input activity j, 



for each service activity j, 



for each service processor k. 



(B.12) 



(B.13) 

The dual LP (B.9)-(B.13) is obtained by perturbing the objective func- 
tion coefficients of the dual static planning problem (3.6)-(3.10). Because 
the dual static planning problem (3.6)-(3.10) has a unique optimal solu- 
tion, for sufficiently small b> 0, the optimal solution of the dual LP (B.9)- 
(B.13) equals {y*,z*) [cf. Mangasarian (1979), Theorem 1]. Therefore, the 
primal problem (B.4)-(B.8) has an optimal solution {p,x). Now choose r 
large enough such that {K^x — Rx\ <be/2. Then R^x >be/2. Consider the 
problem (B.4)-(B.8) with b replaced by 6/2 and R replaced by R^ . Because 
(p,x) is a feasible solution, the optimal objective value < p. The corre- 
sponding dual problem of this new LP is the dual problem (B.9)-(B.13) with 
b in the objective function coefficients replaced by 6/2 and R replaced by 
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W . The optimal objective value of this new dual LP equals ff < p. The new 
dual LP has the exact same constraints as the dual static planning prob- 
lem (3.6)-(3.10) for the rth network. Thus, {y'^,z^) is a feasible solution to 
the new dual LP. It then follows that 

Y: zi + h/2Yyl<f<P- 

This implies that J2iyi ^ 2/)/6 for large enough r, so {y^} is bounded. 

Then we only need to show that every convergent subsequence of {{y'^ ,z^)} 
converges to {y* ,z*). Let (y, z) be a limit point of any subsequence {{y^'" , z^")} 
We will verify that (y, z) is an optimal solution to the dual static plan- 
ning problem (3.6)-(3.10) of the limiting network. First, we show that they 
are feasible. Since {(y''", z^")} — > {y,z) as n — > oo, for any e > 0, for large 
enough n, \y — y**"! < e, \z — z^" \ < e and \R — R^^l < e. For each input ac- 
tivity i G J/, 

yi^ij ^ E ^r^r; + (\y\ + ^^p l^' 



<-Y ^kjzl" + leMyl + sup 




Since e can be arbitrarily small, we have 

Y^ViRij ^ ~ Y^ Akj^k for each input activity j € J^i- 

Similarly, one can verify that 

Y^y^^ij — Y^ ^kjZk for each service activity j E Js 

and 

Y ^k = l- 

keKs 

Furthermore, because (?/''" , z^" ) are optimal solutions, J2keK:i -^fc" ~ P^" ■ Again, 
we can show that 

keKi 

Therefore, {y,z) is an optimal solution to the dual problem (3.6)-(3.10) 
of the limit network. Then by the uniqueness of the optimal solution, we 
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conclude that (y, -2) = {y*,z*). Since the subsequence is arbitrary, we have 
{y^,z^) ^ {y*,z*) as r ^ oo. □ 

Proof of Lemma 10. Similar to the proof of Lemma 1 above, we can 
prove that x'^ ^ x* as r ^ oo. From the strict complementary theorem [cf. 
Wright (1997)], every pair of primal and dual LPs has a strict complementary 
optimal solution if they have optimal solutions. Hence, we have the following 
relations: for the limit network, 

(B.14) ^ AkjX* = 1 is equivalent to 4 > foi' ^ ^ ^s] 

i&Js 

(B.15) ^^y*Rij = ^ ^kjZk is equivalent to x* > for all j G Js] 

(B.16) ^ y^Rij = — ^ ^kjzl is equivalent to x* > for all j G Jj; 
and for each r, 

(B.17) 4 > implies ^ A^jx^j = for all k G /C5; 

3&Js 

(B.18) x^ > implies Y.y^^j = E ^^i^ for all j G Js] 

iei keJCs 

(B.19) x^j > implies J^Vi^ij = " E ^^jzl for ah j G Ji. 

iai k&Kj 

Since x'' — > x* as r ^ 00, we have for large enough r, 

(B.20) X* > imphes x^ > for all j G J. 

This, together with (B.15) and (B.18), implies that, for large enough r and 
each service activity j G Js-, 

(B.21) Y.y*Rij= J2 Akjzl implies Y,yiRlj= E ^kjzl 

Similarly, it follows from (B.16), (B.19) and (B.20) that, for large enough r 
and each input activity j £ Ji, 

(B.22) J2y^^iJ = - E ^kjz*k implies J^yi^ij = " E ^kjzl 

Suppose zl=0 for some k G /C5, then J2j£js Akjx* < 1. There exists an 
e > such that J2jeJs ^kjXj = 1 — e. For large enough r, we have J2j£jg ^fej^^j < 
J2jeJs "^kjXj + e/2 < 1 — e/2 because x^ — > x* as r — > 00. This implies 4 ~ ^ 
for large enough r. Therefore, for large enough r, 

(B.23) 4 = implies 4 = for all /c G /C5. 
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Because {y*,z*) is the optimal solution to the dual static planning prob- 
lem (3.6)-(3.10), we have for each a £ £, 



(B.24)^ iY.ylR^^a, < E f E - E f E 

For the second inequality, we use the fact that J2jejg ^kjaj < 1 for all 
k G ICs, EjeJi'^kjaj = 1 for ah k G /C/, and EkeJCs4 = llk&Ki^ = 1- 
Since a* £ £* , y* ■ Ra* = max^ef y* ■ Ra = 0. It follows that both inequalities 
in (B.24) are equalities for a*. Therefore, 

(B.25) Y^yi^ii = E ^'^i^fc all ^ '^s with a* > 0, 

iel fcG/Cs 

(B.26) E yi^^i = - E ^fci^fc all e wltli «i > 0' 

(B.27) E = 1 foi' all /c G /C5 with z| > 0. 

From (B.21) and (B.25), we have for large enough r, 

E yi^^j = E fo'^ aU j G with a* > 0. 

iex fce/Cs 

Therefore, 

(B.28) E '^*jYyiKj= E '^j E ^kjzl for large enough r. 

j&Js iel j<^Js fcG/Cs 

Similarly, it follows from (B.22) and (B.26) that, for large enough r, 

(B.29) E E y^^J = - E «1 E A>^^4- 

jeJi i£i jeJi fce/C/ 

From (B.23) and (B.27), it follows that, for large enough r, 



E a*Akj = 1 for ah k e ICs with 4 > 0- 



Therefore, 



(B.30) E ^fe E "-j^kj = E ^fc lai'ge enough r. 

It follows from (B.28) and (B.29) that 

E E^r^ii = E E ^fci^fc - E E ^kjzi 

= 1 — for large enough r. 
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The second equality follows from (B.30), J2keiCs ~ J2jeji (^*j^kj = 1 ^^i 
each keKi, and Efce^c, 4 = P""- 

Then Lemma 10 follows from the fact that maxag^: • Wa = \ — . □ 

Proof of Theorem 1. We define the scaled process X'' via 

X'"(t) = r"2x(r2t) for each t > 0. 

Fix a sample path that satisfies the strong law of large numbers for Uj and 
Let {Z,T) be a fluid limit of (Z ,T ) along the sample path. Following 
the arguments in Section A. 2 in Dai and Lin (2005), such a limit exists and 
satisfies the fluid model equations (6.1)-(6.5) presented in Section 6. Un- 
der the maximum pressure policy with parameter a, each fluid limit {Z,T) 
also satisfies the fluid model equation (6.6). The justification of fluid model 
equation (6.6) is similar to Lemma 4 in Dai and Lin (2005), with the scaling 
r replaced by r^. Therefore, {Z ,T ) is a fluid model solution under the 
maximum pressure policy. Similar to the proof of Theorem 4 in Dai and Lin 
(2005), using a different Lyapunov function f{t) = (a x Z{t)) ■ Z{t), one can 
easily prove that the fluid model under the maximum pressure policy with a 
general parameter set (a,/3) is weakly stable; namely, any fluid model solu- 
tion (Z ,T ) under the maximum pressure policy satisfies Z{t) = for each 
t > given Z{0) = 0. As a consequence, we have for any t > 0, T{t)/t satis- 
fies (3.2)-(3.5) with /? = 1. Because x* is the unique optimal solution to the 
static planning problem (3.1)~(3.5) with objective value equal to 1, T{t) = xt 
for each t > 0. Since this is true for any fluid limit, we have T (t) x*t for 
each t with probability 1, which implies asymptotic efficiency. □ 
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